ftn-A^>«7  /LQnUMENTATI0NPACE  ' 

Form  Approved  1  LJ  j 

OMB  No.  0704  0188 

•  9  B  W  ,v  .•»*;. Tv'  *  "  V*  -^r  ■*  w  •"•■€•  .n.'.t.G«PO  t-'f  t-rnf  *Of  ’r-\. tPw .«q  .r.\...  tiom  Std-  n  .  ‘  *''T  nr  -*•*  s'  ^  ^  S  | 

91111111!  lllllill  &£S&&^5SEBEB8£&8SF*st ! 

[  2.  REPORT  OATE  3.  REPORT  TYPE  AND  DATES  COVERED  | 

1  1  MAY  1993  XJ&iEj&ilSx'DISSERTATION  ...  j 

4.  TITLE  AND  SUBTITLE 

Digital  Tracking  and  Control  of  Retinal  Images 

5.  FUNDING  NUMBERS  j 

\ 

1 

l 

j 

i 

i 

i 

.  J 

6.  AUTHOR(S) 

Steven  Frank  Barrett 

•7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

1  AFIT  student  Attending:  The  Univ  of  Texas  at  Austin 

i 

t 

nTir  - 

8.  PERFORMING  ORGANIZATION  j 

REPORT  NUMBER  j 

AFIT/CI/CIA-  93-O10D  } 

i 

1 

‘  T  1 

? 

.  U.  SPONSORING  MONITORING  AGENCY  NAME'S)  AND  ADDRESS(ES)|^  |  J[  t. 

i  DEPARTMENT  OF  THE  AIR  FORCE  p,  cr'Trr  m 

AFIT/CI  O  tLtL  1  b  3 

f  2950  P  STREET  AUG  6  19938 

WRIGHT-PATTERSON  AFB  OH  45433-7765  £ 

10.  SPONSORING  MONITORING  ; 

AGENCY  REPORT  NUMBER  ^ 

)  i 

.  IT.  SUPPLEMENTARY  NOTES  i 

!  ..  _  | 
) 

V  • 

i  -  -  T  —  _ / 

:  12i.  DISTRIBUTION  AVAILABILITY  STATEMENT 

| 

■  Approved  for  Public  Release  IAW  190-1 

1  Distribution  Unlimited 

t  MICHAEL  M.  BRICKER,  SMSgt,  USAF 

’  Chief  Administration 

12b.  DISTRIBUTION  COL’f 

1  » 

1 

i  13.  ABSTRACT  ximum  ±GO  woi as) 

i 


! 

\ 


!  i-  o  *54  lllil 

I 

> 

» 

18102 

11(111  | 

i  14.  SUBJECT  TERMS 

IS.  NUMBER  OF  PAGES  l 

277  [ 

16.  PKlCi  COOT  * 

1 

C  17.  SECURITY  CLASSIFICATION 

5  or  report 

t 

t 

f 

18.  SECURITY  CLASSIFICATION 

OF  THIS  PAGE 

19  security  classification 
of  abstract 

'“v.“  :lON  i  F  ABSTRACT  j 

> 

1 

NSN  /SiO-Ol-.’RC  SSOO  Sto-a.uo  ,  69 

•  rvvi  ..  Ai  <  *•  h 

DISCLAIM  NOnCE 


THIS  DOCUMENT  IS  BEST 
QUALITY  AVAILABLE.  THE  COPY 
FURNISHED  TO  DTIC  CONTAINED 
A  SIGNIFICANT  NUMBER  OF 
COLOR  PAGES  WHICH  DO  NOT 
REPRODUCE  LEGIBLY  ON  BLACK 
AND  WHITE  MICROFICHE. 


DIGITAL  TRACKING  AND  CONTROL 
OF  RETINAL  IMAGES 


APPROVED  BY 
DISSERTATION  COMMITTEE: 


Supervisor: 


To  Cindy,  thanks  for  being  you. 


DIGITAL  TRACKING  AND  CONTROL 
OF  RETINAL  IMAGES 


by 

STEVEN  FRANK  BARRETT,  B.S.,  M.E. 

DISSERTATION 

Presented  to  the  Faculty  of  .he  Graduate  School  of 
The  University  of  Texas  at  Austin 
in  Partial  Fulfillment 
of  the  Requirements 
for  the  Degree  of 

DOCTOR  OF  PHILOSOPHY 


THE  UNIVERSITY  OF  TEXAS  AT  AUSTIN 
May  1993 


Acknowledgements 


I  only  wish  I  could  put  into  words  the  truly  heart  felt  thanks  to  all  of  the  friends 
and  family  who  made  this  adventure  possible. 

I  would  like  to  begin  by  thanking  my  committee  members:  Dr.  A.J. 
Welch,  Dr.  Alan  Bovik,  Dr.  Wilson  Geisler,  Dr.  John  Pearce,  Dr.  Rebecca 
Richards-Kortum,  and  Dr.  Grady  Rylander.  I  appreciate  the  well  prepared 
lessons,  the  stimulating  ideas,  and  your  willing  assistance  on  many  occasions. 
A  special  thanks  to  Dr.  Welch  for  your  always  open  door  and  unassuming 
leadership.  You  are  a  great  role  model. 

I  would  also  like  to  thank  Brigadier  General  Erlind  Royer,  Colonel  Alan 
Klayton,  and  Lieutenant  Colonel  Harry  Bare  for  making  this  all  possible.  I 
have  dedicated  myself  to  apply  what  I’ve  learned  to  being  a  better  teacher  and 
officer  in  service  to  the  cadets  at  the  United  States  Air  Force  Academy.  I  would 
also  like  to  thank  Major  Mike  Markow  who  told  me  about  UT  and  this  exciting 
project.  Your  work  has  formed  the  basis  for  this  research  and  many  others. 

At  UT  I  have  made  many  friends.  I  would  like  to  thank  Chris  Humphrey 
for  her  expert  photography  advice  and  Arthur  Birdwell  for  his  technical  advice. 
A  special  thank  you  to  Dr.  Maya  Jerath.  I  really  appreciate  your  insight  and 
friendship.  I  would  also  like  to  thank  Dr.  Jerry  Fenig,  DVM,  and  staff  at  the 
Animal  Resource  Center  for  the  expert  animal  care  and  assistance.  You  were 
instrumental  in  successful  in  vivo  testing. 


IV 


To  my  family  . . .  you’ve  been  wonderful!  Thanks  mom  and  dad  for 
teaching  me  anything  is  possible  if  you  work  hard  and  believe  in  yourself. 
Thank  you  Jackie  and  Ray  for  making  me  a  part  of  your  family.  You  showed 
me  hard  work  is  important  but  so  is  having  fun.  I  wish  you  were  here  Ray 
to  share  this  with  me.  Thank  you  Heather,  Jonathan,  and  Graham  for  your 
understanding  and  patience.  The  little  time  we  were  able  to  spend  together 
made  the  long  work  hours  worthwhile.  I  love  you  three  with  all  my  heart. 
Cindy,  thank  you  for  the  best  17  years  of  my  life.  You  worked  harder  than  I 
did  these  past  three  years  keeping  everybody  happy.  I  love  you. 

This  research  was  sponsored  in  part  by  the  Texas  Coordinating  Board 
and  in  part  by  the  Office  of  Naval  Research  under  grant  N00014-91-J-1564. 


v 


DIGITAL  TRACKING  AND  CONTROL 
OF  RETINAL  IMAGES 

Publication  No.  _ 

Steven  Frank  Barrett,  Ph.D. 

The  University  of  Texas  at  Austin,  1993 

Supervisor:  Ashley  J.  Welch 


Laser  induced  retinal  lesions  are  used  to  treat  a  variety  of  eye  diseases 
such  as  diabetic  retinopathy  and  retinal  detachment.  Both  the  location  and 
size  of  the  retinal  lesions  are  critical  for  effective  treatment  and  minimal  com¬ 
plications.  Currently,  once  an  irradiation  is  begun,  no  attempt  is  made  to  alter 
the  laser  beam  location  on  the  retina.  However,  adjustments  are  desirable  to 
correct  for  patient  eye  movements.  Lesions  form  in  much  less  than  one  second 
and  typical  treatment  for  a  disease  such  as  diabetic  retinopathy  requires  as 
many  as  2000  lesions  per  eye.  This  type  of  tedious  task  is  ideally  suited  for 
computer  implementation. 

A  system  has  been  developed  to  track  a  specific  lesion  coordinate  on 
the  retinal  surface  and  provide  corrective  signals  to  maintain  laser  position  on 
the  coordinate.  Six  distinct  retinal  landmarks  are  tracked  on  a  high  contrast 
retinal  image  using  two-dimensional  blood  vessel  templates.  Use  of  therapeutic 
lesions  as  tracking  algorithm  landmarks  is  also  investigated.  An  X  and  Y 
laser  correction  signal  is  derived  from  the  landmark  tracking  information  and 


vi 


provided  to  a  pair  of  galvanometer  steered  mirrors  to  maintain  the  laser  on 
a  prescribed  location.  Once  the  laser  position  has  been  corrected,  a  function 
checks  the  terminal  laser  position  for  minor  corrections. 

A  development  speed  tracking  algorithm  has  been  implemented  and 
tested  using  both  vessel  and  lesion  templates.  Closed  loop  feedback  control 
of  laser  position  is  demonstrated  with  calibrated  retinal  velocities  and  in  vivo 
testing  of  the  development  system. 

Trade  ofF  analysis  of  parameters  affecting  tracking  system  performance 
is  provided.  The  analysis  is  used  to  specify  requirements  and  implementation 
details  for  a  real  time  system. 
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Chapter  1 
Introduction 


1.1  Overview 

Dr.  A.J.  Welch,  Dr.  H.  Grady  Rylander  III  M.D.,  and  associated  researchers 
have  worked  toward  the  development  and  system  specification  for  a  Robotic 
Laser  System.  The  overall  goal  of  this  ongoing  project  is  development  of  an 
automated  surgical  laser  system  for  placing  laser  lesions  on  the  human  retina 
for  the  treatment  of  retinal  diseases. 

Laser  induced  lesions  are  used  to  treat  a  variety  of  retinal  diseases  includ¬ 
ing  diabetic  retinopathy,  macular  degeneration,  and  retinal  breaks  and  tears. 
During  treatment  for  these  diseases  an  argon  laser  beam  is  directed  into  the 
eye  via  the  cornea.  Due  to  the  optical  characteristics  of  the  cornea,  the  aque¬ 
ous  humor,  lens,  and  the  vitreous  humor;  the  argon  laser  light  passes  through 
these  media  to  the  retina.  The  argon  laser  light  is  absorbed  and  converted  to 
heat  in  the  pigment  epithelium.  Heat  conducted  from  the  pigment  epithelium 
coagulates  the  retinal  tissue.  The  thermally  damaged  retinal  tissue  results  in 
therapeutic  lesions  useful  for  the  treatment  of  retinal  disease.  The  size  of  the 
retinal  lesion  is  critical  for  the  treatment  of  the  diseases  and  minimization  of 
complications. 

Laser  treatment  is  currently  performed  in  a  ballistic  manner.  The  oph- 
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thalmologist  aims  the  laser  at  the  prescribed  retinal  lesion  site  and  then  fires 
the  laser.  The  laser  has  a  preset  power,  spot  size,  and  exposure  time.  Once 
the  laser  is  fired,  no  attempt  is  made  to  compensate  for  variability  in  retinal 
tissue  absorbance  or  for  retinal  movement.  Placing  lesions  in  this  manner  in 
the  correct  location  and  of  the  correct  size  is  an  acquired  art  [1].  Further¬ 
more,  treatment  protocol  for  diabetic  retinopathy  requires  up  to  2,000  laser 
lesions  per  retina.  This  tedious,  yet  critical  task  is  ideally  suited  for  computer 
implementation. 

Markow  [2]  studied  the  feasibility  of  a  Robotic  Laser  System.  His  intent 
w as  to  “develop  an  automated  laser  delivery  system  and  retinal  observation 
system  that  is  capable  of  placing  multiple  lesions  of  predetermined  sizes  into 
known  locations  in  the  retina”.  Markow  demonstrated  the  concept  of  such  a 
system.  However,  considerable  research  was  required  to  advance  the  concept 
to  a  realizable  system. 

The  intent  of  my  research  is  the  development  of  a  software  algorithm 
to  track  and  correct  for  human  retinal  movement  during  robotic  controlled 
laser  treatment  of  various  retinal  diseases.  From  this  general  statement  an 
entire  research  effort  has  grown  to  encompass  retinal  tracking,  Lesion  Data 
Base  building  for  the  treatment  of  diabetic  retinopathy  and  retinal  breaks  and 
tears,  a  laser  pointing  subsystem,  work  on  adaptive  template  mechanisms  to 
track  movements  when  visible  retinal  features  are  not  available  to  serve  as  land¬ 
marks,  trade  off  analysis  of  factors  affecting  tracking  algorithm  performance, 
and  specification  of  a  real  time  system. 

This  chapter  begins  with  a  description  of  the  treatment  protocol  en¬ 
visioned  for  using  a  Robotic  Laser  System.  This  is  followed  by  a  general  de- 


scription  of  the  two  main  systems  that  work  concurrently  to  produce  controlled 
lesions:  a  Reflectance  Based  Feedback  Control  System  and  a  Retinal  Obser¬ 
vation  and  Tracking  System.  A  brief  review  is  provided  on  the  work  of  my 
colleague  Dr.  Maya  Jerath  on  the  former  system.  The  remainder  of  this  docu¬ 
ment  details  my  research  on  the  latter  system.  The  chapter  concludes  with  an 
overview  of  this  research  effort. 

1.2  Treatment  Protocol 

The  following  paragraphs  describe  the  treatment  protocol  envisioned  for  using 
a  Robotic  Laser  System.  This  protocol  combines  treatment  methods  currently 
used  in  retinal  photocoagulative  therapy  with  Robotic  Laser  System  capabili¬ 
ties. 

To  treat  a  patient  with  the  Robotic  Laser  System  requires  a  series  of  ap¬ 
pointments.  During  the  first  appointment  the  patient  has  their  retina  mapped 
using  the  Retinal  Observation  Subsystem.  This  system  uses  a  standard  fun¬ 
dus  camera  connected  to  a  video  charge  coupled  device  (CCD)  camera  and  a 
video  frame  grabber  to  map  the  visible  surface  of  the  retina.  The  video  frame 
grabber  provides  still  ‘snapshots’  of  retinal  movements.  Mapping  of  the  retina 
is  subdivided  into  separate  fundus  camera  fields  of  view  or  the  individual  fields 
of  view  may  be  mosaiced  into  a  single  retinal  map  as  illustrated  in  Figure  1.1. 
After  the  retinal  map  is  complete  the  patient  will  be  allowed  to  return  home. 

From  the  retinal  map  the  ophthalmologist  diagnoses  the  retinal  disease 
and  prescribes  required  treatment.  A  Lesion  Data  Base  for  the  treatment  is 
then  built  along  with  any  required  tracking  templates.  The  lesion  data  and 
the  templates  are  then  stored  in  the  patient’s  data  file  until  the  follow-up 


Figure  1.1:  The  retinal  map  formed  by  a  mosaic  of  individual  fundus  camera 
field  of  view  images.  Seven  separate  images  were  used  to  form  this  image. 

appointments. 

During  the  follow-up  visit(s).  the  patient  receives  laser  treatment.  The 
patient  is  placed  in  a  supine  position  with  his  head  stabilized  to  prevent  move¬ 
ment.  Reference  Figure  1.2.  The  ophthalmologist  may  space  out  the  treatment 
sessions  to  evaluate  the  effectiveness  of  the  treatment. 

After  the  patient  is  comfortable,  the  fundus  camera  is  aligned  and  fo¬ 
cused  on  the  patient's  retina.  Use  of  a  visual  fixation  device  on  the  conjugate 
eye  assists  in  the  initial  field  of  view  alignment  and  also  minimizes  retinal 
movement  during  the  treatment  session. 

After  initial  alignment  is  complete,  the  tracking  algorithm  establishes 
lock  using  a  set  of  blood  vessel  templates.  Therapeutic  lesions  are  then  placed 
on  the  retina  in  the  precise  location  and  size  as  previously  prescribed  by  the 
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Figure  1.2:  The  conceptual  Robotic  Laser  System  [2]. 


ophthalmologist.  A  single  lesion  may  take  up  to  200  ms  to  form  [4].  Therefore, 
treatment  for  a  single  held  of  view  is  accomplished  in  approximately  45  sec¬ 
onds.  When  a  lesion  reaches  its  prescribed  size,  the  system  issues  the  necessary 
commands  to  close  the  laser  shutter  and  move  the  laser  position  to  the  next 
prescribed  lesion  site  [adapted  from  [2,  3]]. 

During  the  treatment  session  the  eye  may  move,  the  patient  may  blink, 
the  tracker  may  lose  lock,  the  patient  may  panic,  or  some  critical  portion  of  the 
system  might  fail.  The  system  must  have  the  capability  to  respond  to  these 
different  contingencies. 


.1 


Figure  1.3:  The  Robotic  Laser  System. 


1.3  System  Description 

The  Robotic  Laser  System  has  two  main  systems:  the  Reflectance  Based  Feed¬ 
back  Control  System  and  the  Retinal  Observation  and  Tracking  System.  A 
system  organizational  chart  is  provided  in  Figure  1.3. 

1.3.1  Reflectance  Based  Feedback  Control  System 

The  Reflectance  Based  Feedback  Control  System  is  a  real  time  system  to  mon¬ 
itor  lesion  growth.  It  uses  a  two-dimensional  reflectance  image  acquired  via 
a  CCD  camera  which  views  lesion  formation  on  axis  with  the  argon  coagulat¬ 
ing  laser.  Reflectance  images  are  acquired  and  processed  as  the  lesion  forms. 
When  parameters  of  the  reflectance  images  meet  certain  preset  thresholds,  the 
laser  shutter  is  closed  A  signal  is  ._.en  issued  to  the  Retinal  Observation  and 
Tracking  System  to  load  the  next  lesion  coordinate  and  redirect  the  laser  to 
the  new  lesion  site.  Jerath  has  demonstrated  real  time  lesion  parameter  control 
in  an  egg  white  model  medium  and  in  vivo  using  cross  bred  pigmented  rabbits 
[4,  5,  6,  7,  8j. 
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1.3.2  Retinal  Observation  and  Tracking  System 

The  Retinal  Observation  and  Tracking  (OT)  System  tracks  a  specific  lesion 
coordinate  on  the  retinal  surface.  The  system  also  provides  corrective  signals 
to  maintain  the  laser  position  on  the  retina  and  assists  the  ophthalmologist  in 
building  a  Lesion  Data  Base. 

The  OT  system  has  been  subdivided  by  function  into  the  following  sub¬ 
systems:  Retinal  Observation  Subsystem  (ROS),  Retinal  Tracking  Subsystem 
(RTS),  and  Laser  Pointing  Subsystem  (LPS). 

Retinal  Observation  Subsystem 

ROS  provides  a  digitized  map  of  the  retina  for  use  in  diagnosis  and  tracking. 
Software  to  automatically  build  a  Lesion  Data  Base  for  treatment  of  diabetic 
retinopathy  and  retinal  breaks  and  tears  resides  in  this  subsystem.  Also,  special 
functions  to  enhance  the  retinal  image,  calculate  statistics,  and  build  a  mosaic 
image  are  all  encompassed  within  this  subsystem. 

Retinal  Tracking  Subsystem 

RTS  tracks  retinal  movement  during  photocoagulation  and  provides  corrective 
signals  to  the  Laser  Pointing  Subsystem  to  maintain  the  laser  on  a  prescribed 
coordinate.  The  tracking  subsystem  responds  to  inputs  from  the  patient  and 
the  Reflectance  Based  Feedback  Contro*  -otem. 
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Laser  Pointing  Subsystem 

LPS  accurately  points  the  laser  at  the  coordinate  provided  by  the  Retinal 
Tracking  System.  The  pointing  system  must  have  the  capability  to  direct  the 
laser  to  any  specified  coordinate  within  the  fundus  camera  retinal  field  of  view. 

1.4  Other  Applications 

The  Retinal  Observation  and  Tracking  System  has  been  specifically  designed 
for  the  treatment  of  various  retinal  diseases.  However,  with  slight  modifications 
this  system  may  be  used  to  stabilize  a  laser  during  corneal  thermal  keratoplasty, 
laser  vessel  welding,  and  other  applications  requiring  stabilized  laser  delivery. 
The  system  may  also  be  used  to  document  the  movement  of  the  retina  for 
physiological  and  psychological  studies.  This  capability  will  be  demonstrated 
later  in  this  document. 

1.5  Preview 

This  document  begins  with  a  brief  review  of  the  anatomy  and  physiology  of  the 
eye  and  retina  pertinant  to  this  research  effort.  Information  is  also  provided 
on  eye  movements, disease,  and  aging.  Each  of  the  different  subsystem’s  design 
and  theory  of  operation  is  then  provided  followed  by  a  discussion  of  the  instru¬ 
mentation  used  to  implement  the  subsystems.  A  chapter  is  also  devoted  to  the 
special  case  of  tracking  on  a  featureless  retina.  System  testing  methodology 
and  results  then  follow.  A  detailed  trade  off  analysis  of  parameters  affecting 
tracking  system  performance  is  then  provided.  This  analysis  is  used  to  discuss 
the  equipment  and  costs  required  to  convert  the  development  system  into  a 
real  time  clinical  system.  This  document  concludes  with  a  review  of  major 
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accomplishments  and  suggested  areas  of  further  research. 


Chapter  2 


The  Eye 


2.1  Eye  Anatomy 

In  this  chapter  a  brief  review  is  provided  on  eye  anatomy,  principal  retinal 
features,  eye  movements,  and  fixation  capability.  This  review  is  followed  by  a 
discussion  of  various  retinal  diseases  and  injuries  potentially  treatable  by  the 
Robotic  Laser  System  and  aging  mechanisms  with  the  eye. 

2.1.1  Gross  Anatomical  Structure 

The  eyes  are  the  complex  sense  organs  of  the  visual  system.  The  principal 
anatomical  structures  are  provided  in  Figure  2.1  [9]. 

The  eye  is  approximately  spherical  (24  mm  long  by  22  mm  across)  [10]. 
It  is  encased  in  a  tough,  protective  outer  layer  called  the  sclera.  The  anterior 
portion  of  the  sclera  is  clear  and  allows  light  to  enter  the  interior  portion  of 
the  eye.  The  clear  scleral  section  is  called  the  cornea.  Light  initially  enters  the 
eye  via  the  cornea  and  then  passes  in  turn  through  the  anterior  chamber,  the 
lens,  and  the  vitreous  chamber.  Light  then  strikes  the  retina. 

The  retina  consists  of  ten  layers.  Within  these  layers  are  the  visual 
receptors:  the  rods  and  the  cones.  There  are  also  four  types  of  neurons  in  the 
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Figure  2.1:  Horizontal  section  of  the  right  eye  [9] 

retinal  layers:  bipolar  cells,  ganglion  cells,  horizontal  cells,  and  the  amacrine 
cells.  The  different  layers  of  the  retina  are  illustrated  in  Figure  2.2  [9]. 

The  rods  and  cones  coupled  with  the  neurons  provide  a  matrix  of  re¬ 
ceptors  with  converging  links  to  the  optic  nerve.  The  rods  and  cones  synapse 
with  the  bipolar  cells,  the  bipolars  cells  synapse  with  the  ganglion  cells,  and 
the  ganglion  cells  converge  to  form  the  optic  nerve.  The  optic  nerve  routes  the 
visual  information  from  the  eye  to  the  occipital  cortex  of  the  brain  [9]. 

2.1.2  Visible  Retinal  Features 

The  retinal  features  visible  by  a  fundus  camera  form  distinct  landmarks  im¬ 
portant  to  the  tracking  task.  The  most  visible  feature  on  the  retinal  surface  is 
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Figure  2.2:  Components  of  the  retina.  C:  cones,  R:  rods,  H:  horizontal  cells, 
and  A:  amacrine  cells  [9] 
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Figure  2.3:  The  visible  retinal  surface  [9] 

the  optic  disk  as  illustrated  in  Figure  2.3  [9]. 

The  optic  disk  is  the  point  at  which  blood  vessels  enter  the  eye  and 
spread  over  the  surface  of  the  retina.  It  is  also  the  point  where  nerves  from  the 
retina  meet  and  exit  the  eye  as  the  optic  nerve  as  discussed  above.  The  exact 
dimensions  of  the  optic  disk  vary  slightly  by  race,  eye,  and  sex.  The  mean 
horizontal  axis  of  the  left  optic  disk  is  1.88  mm  (standard  deviation  0.18)  with 
a  mean  vertical  axis  of  1.77  mm  (standard  deviation  0.19)  [11].  The  right  optic 
disk  has  similar  dimensions.  Similar  values  are  reported  by  Mansour  [12].  Optic 
disk  measurements  axe  used  as  a  calibration  tool  to  measure  the  dimensions  of 
other  retinal  features  in  the  software  developed  for  this  project. 

Near  the  optic  disk  is  the  fovea  which  is  the  area  of  acute  vision  due 
to  its  high  concentration  of  cone  photoreceptors.  The  fovea  is  approximately 
300  microns  in  diameter  and  it  must  be  protected  from  damage.  A  single  laser 
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pulse  to  the  fovea  can  result  in  permanent  degradation  of  acute  vision  [1].  The 
center  of  the  fovea,  the  foveola,  is  located  3.42  mm  (±  .34  mm)  temporally  and 
inferiorly  to  the  horizontal  axis  of  the  optic  disk  [13].  This  dimension  is  used 
in  the  Lesion  Data  Base  building  algorithm.  The  macula  with  a  5000  micron 
diameter  surrounds  the  fovea. 

The  retinal  vessel  network  surrounding  the  optic  disk  and  fovea  is  called 
the  arcades.  It  is  also  visible  from  the  retinal  surface.  Retinal  vessels  range  in 
size  from  50  microns  to  250  microns  [14]. 

2.1.3  Eye  Movements 

Eye  movement  is  controlled  by  the  external  ocular  muscles.  These  muscles 
include  the  lateral  rectus  muscles  for  looking  to  the  side,  the  medial  rectus 
muscles  for  looking  toward  the  nose,  the  superior  rectus  muscle  for  looking  up, 
the  inferior  rectus  muscle  for  looking  down  and  the  superior  and  inferior  oblique 
muscle  for  depressing  and  elevating  the  gaze  respectively  [9].  These  muscles  act 
in  a  coordinated  fashion  to  affect  the  different  types  of  eye  movement.  Certain 
eye  movements  suffer  age  related  degradation.  Generally,  eye  movements  slow 
with  advancing  age.  The  following  paragraphs  detail  the  different  types  of  eye 
movements  important  to  this  study. 

Saccades 

Saccadic  eye  movements  are  rapid  with  velocities  of  up  to  800  degrees  per 
second  for  visual  target  acquisition.  These  movements  rapidly  propel  the  point 
of  visual  fixation  from  one  target  to  another  in  the  visual  field.  They  are 
typically  of  short  duration,  lasting  from  20  to  200  milliseconds,  and  are  ballistic 
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in  nature.  Most  naturally  occurring  saccades  are  less  than  15  degrees.  For 
larger  saccadic  eye  movements  the  head  may  be  moved  [15]. 

Smooth  Pursuit  and  Vergence  Movements 

Smooth  pursuit  and  vergence  movements  are  for  tracking  movements  to  follow 
a  slowly  moving  object.  These  movements  maintain  the  image  of  the  moving 
object  on  the  fovea  of  ch  e\  .  The  vergence  system  maintains  the  image  on 
the  fovea  as  an  object  Ok.  es  toward  or  away  from  the  observer  and  the  smooth 
pursuit  system  tracks  objects  with  horizontal  or  vertical  movement.  Both  of 
these  systems  are  slow  compared  to  the  saccadic  system.  The  smooth  pursuit 
system  can  accurately  track  up  to  approximately  50  degrees  per  second  [15]. 

Optokinetic  and  Vestibular- Ocular  Movements 

The  optokinetic  and  the  vestibular-ocular  systems  are  used  to  compensate  for 
observer  motion.  These  systems  maintain  stable  vision  as  a  person  moves. 
These  systems  work  together  to  provide  accurate  compensation  for  head  move¬ 
ment  over  a  wide  range  [15]. 

Micro-saccades  and  Micronystagmus  Movements 

Micro-saccadic  movements  are  required  to  maintain  visibility  of  stationary  ob¬ 
jects  due  to  image  fading.  These  small  movements  occur  approximately  every 
second  and  shift  the  gaze  by  5  to  10  minutes  of  arc.  These  movements  are 
difficult  to  suppress.  Micronystagmus  are  oscillatory  movements  at  rates  of 
approximately  0.02  Hertz  and  amplitudes  up  to  approximately  1  minute  of  arc. 
From  an  engineering  point  of  view  these  movements  may  be  regarded  as  system 
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noise  [16]. 

2.1.4  Retinal  Tracking  System  requirements  with  reference  to  reti¬ 
nal  movement 

There  are  many  factors  which  affect  the  Retinal  Tracking  Subsystem’s  capa¬ 
bility  to  maintain  ‘lock’  during  retinal  movement.  These  factors  are  discussed 
in  Chapter  8.  The  two  most  important  factors  are  retinal  velocity  and  desired 
Laser  Pointing  Subsystem  target  radius.  Target  radius  is  defined  as  the  radius 
of  a  circle  which  contains  laser  spot  centroid  movement.  Figure  2.4  illustrates 
the  number  of  position  updates  required  per  second  for  a  given  retinal  velocity 
to  achieve  a  desired  target  radius. 

2.1.5  Eye  Fixation  Capability 

Different  fixation  methods  may  be  used  to  minimize  eye  movements.  These 
include:  bite  bars,  stabilizing  cushions,  chin  or  head  stabilizers,  or  a  visual 
fixation  device.  These  methods  may  be  used  separately  or  in  combination. 
For  this  study  a  visual  fixation  array  of  light  emitting  diodes  has  been  used  in 
combination  with  a  chin  and  forehead  stabilizer  to  minimize  patient  eye  move¬ 
ments.  A  detailed  description  of  the  visual  fixation  device  design  is  provided 
later  in  this  document. 

Human  visual  fixation  is  provided  by  a  negative  feedback  mechanism. 
This  mechanism  prevents  the  point  of  visual  fixation  from  leaving  the  area  of 
the  fovea  on  the  retinal  surface.  When  a  spot  of  light  is  focused  on  the  fovea 
micronystagmus  movements  cause  the  spot  to  move  back  and  forth  across  the 
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target  radius  (microns) 


Figure  2.4:  Factors  influencing  the  Retinal  Tracking  Subsystem’s  ‘lock’  capa¬ 
bility.  The  number  of  position  updates  required  per  second  are  provided  as  a 
function  of  retinal  velocity  in  degrees  per  second  and  desired  Laser  Pointing 
Subsystem  target  radius. 
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cones.  Each  time  the  spot  reaches  the  edge  of  the  fovea  a  micro-saccade  occurs 
bringing  the  spot  back  to  the  central  foveal  region  [17].  Many  movements  of 
these  types  have  been  observed  during  the  course  of  this  study.  Studies  con¬ 
ducted  by  Kosnik,  Fikre,  and  Sekuler  with  untrained  psychophysical  observers 
indicate  that  fixation  stability  does  not  degrade  significantly  with  age.  They 
define  fixation  stability  using  a  contour  ellipse  of  the  scatter  of  eye  positions 
about  its  mean  position.  The  area  of  the  ellipses  is  expressed  in  minutes  of 
squared.  The  young  group  (mean  age  =  22  years)  have  a  mean  ellipse  area  of 
165  min  of  arc  (S.D.  90.2)  while  the  older  group  (mean  age  =  70  years)  have  a 
mean  ellipse  area  of  198  min  of  arc  (S.D.  90.4)  [18]. 

2.2  Diabetic  Retinopathy  and  Other  Retinal  Diseases 

The  Robotic  Laser  System  will  have  the  capability  to  treat  various  eye  diseases 
and  injuries.  Some  of  the  more  common  maladies  treatable  by  the  Robotic 
Laser  System  are  described  in  this  section. 

2.2.1  Diabetic  Retinopathy 

Diabetic  retinopathy  is  a  disease  of  the  retina  that  begin®  a  n^n-inflammatory 
role  and  progresses  through  increasingly  severe  stages.  The  non-inflammatory 
stage  is  characterized  by  small  aneurysms  and  hemmorhages  along  the  retinal 
surface.  The  preproliferative  stage  is  characterized  by  blood  vessel  obstruction. 
The  final  and  most  severe  stage  is  called  the  proliferative  stage.  The  key  feature 
of  the  proliferative  stage  is  the  rapid  formation  of  new,  poor  quality  blood 
vessels.  This  characteristic  is  called  neovascularization.  These  new  vessels 
grow  into  the  vitreous  portion  of  the  eye  and  may  obstruct  the  visual  path. 


Also,  these  poorly  formed  vessels  leak  blood  into  the  vitreous  chamber  further 
obstructing  vision  [1]. 

The  precise  stimulus  for  neovascularization  is  unknown.  In  1956  Wise 
[19]  hypothesized  a  retinal  hypoxic  condition  stimulated  the  new  vessel  growth. 
His  hypothesis  has  yet  to  be  confirmed.  However,  recent  work  by  Stefansson 
et  al.  support  Wise’s  hypothesis.  Their  experiments  demonstrate  that  oxy¬ 
gen  tension  was  significantly  higher  over  retinal  areas  treated  with  panretinal 
photocoagulation  than  other  untreated  areas  of  the  same  retina  [20]. 

To  better  understand  the  treatment  protocol  for  diabetic  retinopathy 
a  closer  examination  of  the  retinal  oxygen  supply  is  required.  The  retinal 
oxygen  supply  is  provided  by  two  separate  systems:  1)  the  inner  retinal  supply 
providing  oxygen  from  the  vitreous  to  the  outer  plexiform  layer  and  2)  the 
outer  retinal  supply  providing  the  needs  of  the  avascular  photoreceptors(the 
rod  and  cones).  The  inner  retinal  supply  is  provided  by  the  retinal  circulatory 
system  while  the  outer  retinal  supply  is  provided  by  diffusion  from  the  choroidal 
circulation. 

The  retinal  circulatory  system  is  sensitive  to  changes  in  the  oxygen 
supply.  A  hypoxic  condition  induces  an  autoregulatory  vasodilation  response. 
The  retinal  vessels  adjust  their  flow  to  maintain  the  tissue  oxygen  level  at  a 
constant  level.  The  retinal  circulatory  system  provides  50  percent  of  its  oxygen 
to  the  tissue.  The  choroidal  system  does  not  autoregulate  significantly.  Only 
4  percent  of  its  oxygen  supply  is  provided  to  the  retinal  tissue  [21]. 

Homeostasis  of  oxygen  availability  in  the  retina  provides  a  sufficient 
mechanism  to  initiate  or  inhibit  vessel  growth.  Dilation  of  retinal  vessels  for 
any  length  of  time  initiates  new  vessel  growth.  The  rate  of  new  vessel  growth 


is  proportional  to  the  amount  of  dilation  of  retinal  vessels.  The  loss  in  diabetes 
of  the  sphincter-like  mural  cells  may  facilitate  retinal  vessel  dilation  [21]. 
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Diabetic  retinopathy  is  treated  with  panretinal  photocoagulation.  An 
argon  laser  at  a  wavelength  of  488  nanometers  and  514  nanometers  is  used  to 
selectively  denature  peripheral  portions  of  the  retina  while  sparing  critical  vi¬ 
sion  anatomy  about  the  fovea  and  optic  disk.  The  retinal  vessel  network  is  also 
spared.  Many  different  lesion  patterns  may  be  used.  A  pattern  of  two  concen¬ 
tric  rings  of  200  micron  lesions  about  the  critical  vision  anatomy  surrounded  by 
concentric  rings  of  500  micron  lesions  out  to  the  far  retinal  periphery  is  com¬ 
mon  [1].  This  technique  preserves  acute  vision  about  the  macula  at  the  expense 
of  the  peripheral  vision.  This  treatment  is  based  on  the  hypothesis  that  the 
lesions  selectively  destroy  rods  and  cones  by  photocoagulation  to  allow  more 
choroidal  oxygen  to  reach  the  inner  retina  and  constrict  retinal  vessels.  This 
selective  denaturation  improves  the  oxygen  supply  to  the  retina  by  increasing 
the  oxygen  tension.  This  has  recently  been  verified  in  human  patients  [20].  The 
improved  oxygen  tension  suppresses  the  neovascularization  response.  The  suc¬ 
cess  of  argon  laser  treatment  is  roughly  proportional  to  the  amount  of  retinal 
tissue  photocoagulated  [21]. 

2.2.2  Macular  Degeneration 

Macular  degeneration  is  also  called  senile  macular  degeneration  because  it  is 
most  common  in  the  elderly  population.  This  disease  is  the  leading  cause  of 
blindness  in  people  over  65  years  of  age.  However,  the  disease  may  also  affect 
younger  people.  This  disease  occurs  in  two  forms:  the  more  common  drusenoid 
or  dry  form  and  the  neovascularization  or  exudative  form.  The  neovascular- 
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ization  form  occurs  in  approximately  20  percent  of  the  macular  degenerative 
cases.  This  is  the  more  active  of  the  two  forms.  This  form  can  be  tre  i  via 
argon  laser  photocoagulation  techniques  while  the  drusenoid  form  can  not.  The 
neovascularized  form  is  typified  by  leaky  blood  vessels  and  hemmorhaging  into 
the  macula  and  the  fovea.  Treatment  is  similar  to  that  prescribed  for  diabetic 
retinopathy.  In  about  10  percent  of  the  neovascularized  cases,  the  bleeding  is 
so  close  to  the  fovea  that  treatment  is  not  currently  possible.  If  untreated,  the 
fovea  may  become  obliterated  and  destroyed  within  a  month  or  two  resulting 
in  loss  of  acute  vision  [22]. 

2.2.3  Retinal  Breaks  and  Tears 

The  retina  may  be  subdivided  into  two  main  layers:  the  neural  retina  and  the 
retinal  pigment  epithelium.  The  pigment  epithelium  provides  a  nursing  role  to 
the  rods  and  the  cones.  Certain  traumatic  injuries  result  in  retinal  breaks  and 
tears.  If  left  untreated  the  two  layers  may  separate.  This  separation  is  called 
retinal  detachment.  If  the  layer  separation  is  not  repaired  blindness  will  result. 
The  retinal  breaks  and  tears  may  be  repaired  using  photocoagulation  to  seal 
the  break.  The  rods  and  cones  within  the  traum..  site  are  no  longer  functional 
[1,  53].  A  common  technique  is  to  surround  the  torn  area  with  two  continuous, 
concentric  rings  of  200  micron  lesions. 


2.3  The  Aging  Eye 

It  is  important  to  review  the  effects  of  aging  on  the  eye  since  many  patients 
treatable  by  the  Robotic  Laser  System  will  be  elderly.  Weale  [23]  has  carefully 
documented  the  effects  of  aging  on  different  eye  structures.  The  effects  include: 


•  The  cornea  yellows  in  advanced  age.  Also,  the  older  cornea  tends  to 
scatter  more  light  [23]. 
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•  There  is  a  marked  decrease  in  pupil  area.  Weale  notes  the  ratio  of  maxi¬ 
mum  to  minimum  pupillary  area  slowly  decreases  with  age  [23]. 

•  The  crystalline  lens  tends  to  scatter  more  light  [23]. 

•  The  vitreous  body  usually  has  a  clear,  gel  consistency.  With  advanced 
age  the  collagen  fibrillar  network  within  the  gel  tends  to  agglomerate  and 
form  a  floating  ‘powder’  [23]. 

•  The  retina  may  experience  the  appearance  of  blood  vessels,  yellowish- 
white  spots,  and  drusen  [23].  Drusen  is  hyaline  excresences  in  the  eye 
due  to  aging  [24]. 


Chapter  3 


The  Retinal  Observation  Subsystem 

3.1  Objective 

The  Retinal  Observation  Subsystem  provides  a  digitized  map  of  the  patient’s 
retina  for  use  by  the  ophthalmologist  to  diagnose  different  retinal  diseases  or 
injury  and  plan  required  treatment.  The  tracking  algorithm  also  requires  a 
high  contrast  digitized  image  of  the  retina  where  features  are  clearly  discernible 
against  the  retinal  background.  This  chapter  provides  a  brief  review  of  research 
activity  in  the  area  of  retinal  imaging  followed  by  a  comparative  study  of  dif¬ 
ferent  imaging  systems  considered.  This  chapter  concludes  with  a  description 
of  software  written  to  capture,  enhance,  and  store  the  digitized  retinal  map. 

3.2  Previous  Work  on  Retinal  Imaging 

This  section  provides  a  review  of  techniques  used  to  image  and  enhance  the 
retinal  image  via  optical  filters  and  video  frame  grabber  input  look  up  table 
manipulations. 

3.2.1  Television  Ophthalmoscopy 

Video  funduscopy  is  the  use  of  video  camera  equipment  to  film  the  fundus  - 
“the  back  portion  of  the  interior  of  the  eye  visible  through  the  pupil  [24]”. 
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This  technique,  also  known  as  television  ophthalmoscopy,  is  not  new.  In  1962 
West  et  al.  claimed  “The  idea  of  television  ophthalmoscopy  is  not  a  new  one. 
Ridley  demonstrated  such  a  device  ten  years  ago.”  Early  development  of  this 
technique  was  hindered  by  video  camera  imaging  technology  [25],  Even  with 
these  limitations,  retinal  studies  using  monochromatic  illumination  at  different 
wavelengths  were  accomplished. 

3.2.2  Fundus  Chromatic  Studies 

A  detailed  analysis  of  retinal  monochromatic  response  was  conducted  by  De- 
lori  et  al.  in  1976.  Delori  found  that  specific  features  of  the  fundus  could  be 
imaged  with  increased  contrast  when  appropriate  monochromatic  illumination 
was  used.  Delori  concluded  monochromatic  studies  permit  more  accurate  visu¬ 
alization  than  can  be  achieved  with  white  light.  Important  to  this  investigation 
was  the  conclusion:  “The  optical  wavelength  for  observing  large  retinal  vessels 
is  570  nanometers,  but  their  visibility  is  generally  excellent  between  540  and 
580  nanometers.  The  vessels  appear  dark  and  well  defined  with  a  central  ir¬ 
regular  streak  of  light  along  the  larger  vessels[26]”.  The  improved  visibility  of 
vessels  in  this  region  is  due  to  the  peak  of  hemoglobin  absorption  near  570  nm. 
Reference  Figure  3.1  [27]. 

Based  on  this  conclusion  a  568  nm  interference  filter  was  chosen  to 
enhance  the  retinal  vessel  contrast.  The  results  of  using  this  filter  compared  to 
a  white  light  source  and  a  530  nm  bandpass  filter  is  illustrated  in  3.2.  The  530 
nm  filter  was  provided  as  an  option  with  the  fundus  camera. 
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Figure  3.2:  Retinal  imaging  with  optical  filters.  Left  to  right:  white  light 
source,  530  nm  bandpass  filter,  568  nm  interference  filter 
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3.3  Retinal  Imaging 

This  section  contains  a  description  of  different  imaging  technologies  considered 
for  this  project.  A  brief  theory  of  operation  is  provided  with  each  technique 
along  with  inherent  advantages  and  disadvantages.  This  section  concludes  with 
a  comparative  analysis  and  a  selection  of  an  imaging  system  for  the  remainder 
of  the  study. 

3.3.1  Fluorescein  Angiogram  Retinal  Imaging 

Fluorescein  angiogram  retinal  imaging  is  a  clinical  tool  used  by  ophthalmolo¬ 
gists  to  obtain  a  high  contrast  image  of  the  retina.  Specifically,  it  is  used  to 
image  microaneurysms  associated  with  critical  retinal  diseases  long  before  the 
disease  becomes  acute.  This  technique  may  also  be  used  to  examine  retinal 
vessel  disorders  such  as  blockages  and  leaking.  The  literature  further  indi¬ 
cates  this  technique  may  be  used  to  quantify  blood  flow  parameters  within  the 
retina.  The  motivation  for  studying  this  technique  was  to  improve  the  gray 
level  contrast  between  the  retinal  vessels  and  background  [28]. 

Fluorescein  Angiogram  Development 

The  fluorescein  angiogram  technique  is  not  new.  It  was  originally  introduced  to 
clinical  practice  by  Novotny  and  Alvis  in  1961.  Their  objective  was  to  develop 
a  technique  to  observe  retinal  blood  flow  with  increased  visibility  and  defini¬ 
tion.  They  originally  determined  both  the  excitation  and  emission  wavelengths 
of  blood  fluorescein  mixtures  spectrofluorometrically.  The  optimal  excitation 
wavelength  was  found  to  be  490  nm  and  the  maximal  emission  wavelength  was 
520  nm.  Blue  and  green  filters  were  used  to  modify  the  excitation  and  emission 
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paths  respectively  [29]. 

Fluorescein  Angiogram  Current  Techniques 

Currently,  fluorescein  imaging  is  accomplished  by  injecting  fluorescein  sodium 
10  percent  in  a  dose  of  10  mg  per  kg  body  weight  as  a  bolus  into  a  cubital  vein. 
The  time  from  injection  into  the  vein  until  fluorescence  visualization  varies  from 
12  to  30  seconds.  The  fluorescence  lasts  for  approximately  3.5  minutes.  The 
fluorescein  sodium  eventually  appears  in  the  aqueous  humor  which  contributes 
to  the  loss  of  image  clarity  [29,  28]. 

A  fundus  camera  equipped  with  a  standard  35  mm  camera  is  typically 
used  to  film  the  progress  of  the  fluorescein  sodium  in  the  retinal  vessels.  A 
blue  filter  with  a  passband  similar  to  the  fluorescein  sodium  excitation  band  is 
placed  in  the  illuminating  source’s  path.  Ideally,  this  excitation  filter  should 
have  a  passband  from  400  to  500  nm.  However,  the  absorption  characteristics 
of  the  human  eye  from  400  to  450  nm  severely  limit  excitation  at  these  wave¬ 
lengths.  A  barrier  filter  with  a  passband  similar  to  the  emission  band  is  placed 
before  the  camera’s  objective  lens.  Ideally,  this  is  a  highpass  filter  which  blocks 
wavelengths  below  500  nm  [28]. 

Advantages  of  the  Fluorescein  Angiogram 

The  fluorescein  angiogram  allows  the  discernment  of  small  structures  and  tem¬ 
poral  studies  of  the  retinal  vessels  not  possible  with  other  imaging  techniques. 
In  fact,  Nielsen  noted  that  fluorescein  sodium  is  the  most  employed  dye  in  oph¬ 
thalmology  for  diagnostic  purposes  [28].  A  sample  of  a  fluorescein  enhanced 
retinal  image  is  provided  in  Figure  3.3.  The  image  was  provided  by  Dr.  H. 
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Figure  3.3:  Fluorescein  enhanced  retinal  image 


Grady  Rylander. 

The  Disadvantages  of  Fluorescein  Sodium 

The  adverse  affects  of  using  fluorescein  sodium  imaging  are  well  documented. 
Thorough  studies  accomplished  in  1982  and  1983  on  several  thousand  patients 
indicated  complications  occur  in  5.4  percent  of  cases.  The  study  indicated  that 
complications  tend  to  occur  more  frequently  in  male  patients  who  have  had 
multiple  angiographies.  The  most  common  ad verse  effect  was  transient  nausea. 
The  study  estimated  that  life  threatening  circulatory  reactions  occur  in  5  out 
of  10,000  angiographies.  However,  it  was  noted  that  serious  cardiopulmonal 
complications  may  be  coincidental  only  as  a  reflex  mechanism  associated  with 
venipuncture  and  not  with  the  injected  dye.  The  study  concluded  that  “despite 
the  low  incidence  of  life  threatening  reactions  fluorescein  angiography  of  the 
retina  should  be  performed  only  when  the  indication  is  justified  and  then  only 
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Figure  3.4:  Scanning  Laser  Ophthalmoscope  [31] 


if  it  provides  a  guideline  for  treatment  [28]”. 

3.3.2  Scanning  Laser  Ophthalmoscope  (SLO)  Imaging 

The  SLO  technique  of  retinal  imaging  uses  a  dim  568.2  nm  krypton  laser  beam 
to  scan  the  retina.  Recall  from  Delori’s  study  that  568.2  nm  is  close  to  the 
peak  absorption  of  hemoglobin. 

There  are  two  main  optical  paths  in  the  SLO  system:  1)  the  raster 
path,  and  2)  the  light  collection  path.  The  raster  optical  path  provides  the 
vertical  and  horizontal  sweeps  of  the  laser  light  source.  The  sweeps  are  ac¬ 
complished  using  acousto-optical  modulators  (AOM)  and  mirrors  connected 
to  galvanometers.  Basically,  the  intensity  modulated  beam  from  the  AOM  is 
swept  vertically  in  a  sawtooth  waveform  by  a  mirror  mounted  on  a  galvanome- 
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Figure  3.5:  SLO  image  at  a  50  degree  field  of  view 


ter  (VMG).  Reference  Figure  3.4  [31].  A  second  mirror  mounted  to  a  tuned 
resonant  galvanometer  (HMG)  sweeps  the  beam  horizontally  with  a  sinusoidal 
waveform.  The  vertical  and  the  horizontal  sweeps  of  the  beam  by  the  mirror 
galvanometers  produce  a  raster  pattern  of  parallel  horizontal  lines  directly  on 
the  retina.  The  returned  light  from  a  specific  point  on  the  retina  is  captured  by 
a  photomultiplier  tube  (PMT)  and  displayed  as  the  intensity  of  the  spot  on  a 
television  monitor.  This  is  accomplished  via  a  small  mirror  (M)  optically  con¬ 
jugate  with  the  eye’s  pupil  and  brought  to  a  focus  on  the  retina  by  an  aspheric 
ophthalmoscopic  lens  (AL).  The  laser  moves  over  the  retina  synchrounously 
with  the  spot  on  the  monitor  such  that  there  is  a  one-to-one  correspondence 
between  a  specific  point  on  the  retina  and  a  specific  point  on  the  television 
screen.  Thus,  a  video  image  is  built  up  point-by-point  [30,  31].  Figure  3.5 
provides  a  sample  of  the  fine  resolution  available  with  the  SLO  technique.  This 
image  was  provided  by  Dr.  Ann  Eisner  of  the  Boston  Eye  Research  Institute. 
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Advantages  of  the  SLO 

The  SLO  imaging  technique  has  many  advantages.  The  advantages  include: 

1.  The  retinal  illumination  required  for  SLO  imaging  is  on  the  order  of  1000 
times  less  than  standard  fundus  imaging  and  10,000  times  less  than  the 
fluorescein  angiogram  [32]. 

2.  The  SLO  uses  only  a  0.9  millimeter  diameter  entrance  pupil  leaving  the 
rest  of  the  pupil  for  the  image.  Since  a  small  entrance  pupil  is  required 
pupil  dilation  is  not  required  [32,  33]. 

3.  The  laser  source  may  be  steered  around  opacities  such  as  cataracts  [32]. 

4.  The  SLO  allows  fluorescein  angiography  with  many  orders  of  magnitude 
less  light  and  one-tenth  the  dye  dosage.  This  allows  the  examination  of 
both  eyes  and  repetitive  examinations  during  a  single  clinical  visit  [32]. 

5.  As  a  patient  ages,  the  lens  and  the  vitreous  humor  tend  to  scatter  and 
absorb  light  in  greater  quantities.  The  scattering  is  seen  as  a  glare  by 
the  patient  and  as  a  cloudiness  by  the  clinician.  The  cloudiness  reduces 
image  contrast.  Increasing  the  level  of  retinal  illumination  in  a  standard 
fundus  camera  further  increases  the  scattering  and  degrades  image  con¬ 
trast.  SLO  reduces  these  scattering  effects  since  retinal  illumination  is 
provided  through  a  smaller  portion  of  the  scattering  medium  [32]. 

6.  Color  images  are  possible  by  using  the  illuminating  laser  in  a  “white  light 
mode”.  In  this  case  there  is  simultaneous  emission  of  647,  568,  and  502 
or  496  nm  light.  Three  separate  detectors  are  required  [32,  33]. 
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7.  Any  graphical  material  that  can  be  displayed  on  a  computer  monitor  can 
also  be  impressed  on  the  retinal  pattern  formed  by  the  sweeping  laser 
beam.  This  capability  would  allow  an  adaptive  feedback  patient  fixation 
device.  This  technique  has  been  used  to  investigate  how  patients  with 
macular  scotomas  (area  of  depressed  vision  [24])  use  residual  functional 
retinal  areas  to  inspect  visual  detail  [32]. 

8.  The  SLO  imaging  system  has  a  large  depth  of  field  which  permits  the 
iris,  vitreous  humor  structures,  and  the  retinal  surface  to  be  in  focus 
simultaneously.  It  also  has  the  capability  to  be  used  in  the  confocal 
mode  where  a  single  retinal  plane  is  in  focus  [32,  33]. 

Disadvantages  of  the  SLO  Technique 

The  disadvantage  with  the  SLO  is  limited  availability  due  to  high  cost. 

3.3.3  Silicon  Intensified  Tube  (SIT)  Cameras 

Silicon  intensified  tube  type  cameras  are  designed  specifically  for  low  light, 
low  contrast  imaging  environments  [35].  The  camera’s  lens  forms  an  image 
on  the  tube’s  photosensitive  element.  The  charge  density  at  a  specific  point 
on  the  element  is  proportional  to  the  incident  light  flux  at  that  point.  An 
electrical  analog  of  the  image  is  thus  formed  on  the  photosensitive  element. 
The  photosensitive  element  is  scanned  with  an  electron  beam  to  convert  the 
charge  distribution  into  a  voltage  [36]. 

Although  these  cameras  work  very  well  in  a  low  light,  low  contrast  envi¬ 
ronment  they  are  heavy,  susceptible  to  damage  from  image  burn,  and  expensive 
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(10,000  dollars)  relative  to  some  of  the  other  camera  technologies  discussed.  A 
SIT  camera  was  tested  early  in  this  study.  Due  to  the  disadvantages  listed 
above  and  the  small  image  obtained  when  coupled  with  a  fundus  camera,  the 
SIT  technology  was  eliminated  from  further  consideration. 

3.3.4  Charge  Coupled  Device  (CCD)  Cameras 

A  charge  coupled  device  camera  has  a  single  integrated  circuit  for  the  aci-ivc 
camera  element.  This  ‘chip1  consists  of  a  two-dimensional  array  of  metal-oxide 
semiconductor  (MOS)  capacitors  operating  in  the  deep  depletion  mode.  The 
capacitor  is  biased  such  that  impinging  light  generates  electron-hole  pairs  that 
are  trapped  in  potential  wells  within  the  MOS  structure.  The  number  of  pairs 
generated  is  proportional  to  the  amount  of  light  impinging  on  a  given  MOS 
capacitor  [37],  Ideally,  each  MOS  device  in  the  array  should  have  a  linear 
response  and  be  identical  to  other  MOS  devices  in  the  array.  This  is  not  the 
case. 

A  charge  coupled  device  generated  image  consists  of  the  entire  array  of 
separate  point  charges.  Each  separate  point  charge  forms  a  picture  element  or 
pixel.  To  convert  the  charge  into  a  useable  signal  it  must  be  shifted  out  a  line 
at  a  time.  Once  shifted  oiu  of  the  array  the  charge  is  converted  to  a  voltage 
signal  a  pixel  at  a  time  until  all  pixels  in  an  image  have  been  converted.  When 
the  shifting  operation  is  complete,  a  new  image  can  be  formed  [37]. 

High  performance  CCD  cameras  are  available  with  dynamic  range  ex¬ 
tending  over  twelve  orders  of  magnitude,  high  sensitivity  down  to  10  micro- 
foot-candles,  and  resolutions  of  750  by  500  pixels  or  more.  To  achieve  this 
dynamic  range  capability  image  intensifies,  and  specialized  iris  and  gating  cir- 
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cuits  must  be  employed  [38].  The  dynamic  range  of  the  CCD  camera  is  limited 
by  generation  and  recombination  noise  on  the  low  intensity  end  and  potential 
well  overload  (or  saturation)  on  the  high  end.  The  CCD  camera  used  in  this 
study  had  sensitivity  of  70  milli-foot-candles,  resolution  of  510  x  492  pixels  and 
operated  at  a  standard  frame  rate  of  30  frames  per  second.  The  camera  costs 
under  one  thousand  dollars. 

3.3.5  Retinal  Image  Enhancement 

There  are  a  number  of  image  processing  techniques  that  may  be  employed  to 
enhance  the  contrast  of  an  image  or  enhance  certain  features  within  an  image. 
These  include  various  filtering  schemes  and  gray  scale  histogram  ir 'edification. 
In  this  study  these  techniques  were  used  with  the  CCD  camera  to  enhance  the 
retinal  image  for  tracking. 

Optical  Filtering 

Ideas  presented  in  Delori’s  work  on  fundus  chromatic  studies  were  used  to 
enhance  the  retinal  image  contrast.  A  568  nm  (Edmund  Scientific  J43,127) 
interference  filter  was  used  inline  with  the  fundus  camera  halogen  illumination 
lamp  to  bathe  the  interior  of  the  eye  in  green  light.  The  result  of  this  filter 
employment  is  provided  in  Figure  3.2  and  characteristics  of  this  filter  are  pro¬ 
vided  in  Figure  3.6.  These  characteristics  were  measured  with  a  Varian  2300 
spectrophotometer. 


36 


Olympus 

Edmund 


Wavelength  (nm) 

Figure  3.6:  J43,127  568  nm  interference  filter  characteristics  contrasted  with 
the  Olympus  530  nm  bandpass  filter 


Real  Time  Histogram  Modification 

The  RETINA  Tracking  and  Image  Analyzer  software  developed  for  this  project 
has  many  capabilities  to  enhance  an  image.  The  specific  method  used  in 
this  study  for  contrast  enhancement  is  histogram  modification  using  function 
mod-his  (modify  histogram)  under  the  Image  Statistics  Menu.  Reference  Fig¬ 
ure  3.7. 

There  are  three  different  options  for  histogram  modification  within  func¬ 
tion  mod-his.  These  functions  have  been  developed  from  Rosenfeld  and  Kak’s 
general  description  of  histogram  modification  techniques  [34].  The  modJhisl 
function  examines  the  gray  level  histogram  of  a  reference  image.  The  refer¬ 
ence  image  is  a  randomly  chosen  image  of  the  patient’s  retina  under  treatment. 
The  function  maps  each  current  gray  level  in  the  reference  histogram  to  a  new 
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single  unique  gray  level.  The  new  gray  level  values  are  equally  spaced  across 
the  gray  level  spectrum  from  0  to  255.  The  gray  levels  between  the  new  gray 
level  values  are  not  used.  This  technique  is  known  as  'global  linear  min-max 
windowing’  [39].  The  gray  scale  modification  performed  by  function  modJiisl 
may  be  described  by: 
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(3-1) 


where: 


•  g modified  gray  level 

•  g:  premodified  gray  level 

•  tv-  lower  bound  of  expanded  histogram  (usually  0) 

•  tn>:  upper  bound  of  expanded  histogram  (usually  255) 

•  tv  lower  bound  of  image  histogram 

•  t^'-  upper  bound  of  image  histogram 

An  illustration  of  this  histogram  modification  is  provided  in  Figure  3.8. 

This  method  was  chosen  because  it  can  be  implemented  on  the  Matrox 
PIP- 1024  frame  grabber  without  a  time  penalty.  This  is  accomplished  by  load¬ 
ing  the  gray  level  mapping  obtained  from  function  mod_hisl  into  the  input 
look  up  table  (ILUT)  of  the  frame  grabber.  All  incoming  video  images  are 
processed  through  the  ILUT.  The  ILUT  normally  has  a  one-to-one  mapping 
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between  gray  levels  of  the  incoming  image  and  the  gray  levels  presented  to 
the  frame  grabber.  However,  this  mapping  function  may  be  modified  to  any 
desired  map  without  incurring  an  additional  time  penalty.  A  sample  obtained 
from  the  histogram  modification  mod_hisl  is  provided  in  Figure  3.9.  Functions 
mod_his2  and  mod  Jhis3  provide  even  more  dramatic  results  but  they  can  not  be 
implemented  without  a  time  penalty.  Many  frame  grabbers  have  the  capability 
to  modify  the  input  look  up  table. 


3.3.6  Imaging  Technique  Comparison 

To  compare  different  imaging  technologies  I  obtained  sample  images  of  the  hu¬ 
man  retina  using  1)  a  CCD  camera  equipped  with  a  568  nm  filter  and  histogram 
modification,  2)  fluorescein  angiogram  images  provided  by  Dr.  H.  Grady  Ry- 
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Figure  3.9:  Real  time  histogram  modification  with  function  mod-hisl 


lander,  and  3)  SLO  images  provided  by  Dr.  Ann  Eisner  of  the  Boston  Eye 
Research  Institute.  The  imaging  technologies  were  compared  on  the  basis  of 
contrast  delta  which  is  the  difference  in  average  gray  level  between  a  vessel 
central  to  the  image  and  the  adjoining  retinal  background.  The  fluorescein  an¬ 
giogram  images  were  found  to  have  the  highest  contrast  delta  (34.5)  followed  by 
SLO  (28.5)  and  the  CCD  with  contrast  enhancement  (23.4).  The  CCD  camera 
image  without  contrast  enhancement  had  a  contrast  delta  of  5.2. 

I  am  reluctant  to  draw  quantitative  conclusions  from  such  a  small  num¬ 
ber  of  samples  (12).  Furthermore,  a  direct  comparison  between  imaging  types 
was  not  possible  since  the  same  subject  eye  was  not  compared  under  similar 
conditions.  Due  to  the  availability  of  the  CCD  camera  and  the  contrast  en¬ 
hancement  provided  by  optical  filtering  and  histogram  modification,  the  CCD 
technology  was  used  for  the  remainder  of  the  study.  The  cost  of  the  SLO  and 
the  concerns  related  to  fluorescein  sodium  limited  further  consideration  of  these 
technologies.  However,  the  tracking  algortihm  may  be  easily  adapted  for  use 
with  either  technology. 

3.4  Storage  Media 

Since  the  Retinal  Observation  Subsystem  will  be  used  in  clinical  practice  it  is 
important  that  pertinant  patient  information  such  as  key  diagnostic  and  treat¬ 
ment  images  be  retained  for  future  reference.  A  single  image  of  512  x  512  pixel 
spatial  resolution  by  256  gray  scale  resolution  requires  256  kilobytes  for  storage. 
A  single  retina  may  require  more  than  sixteen  of  these  images.  A  record  of  both 
eyes  would  thus  require  at  a  minimum  8  megabytes  of  data.  Additional  data 
storage  requirements  which  will  be  discussed  later  in  this  document  account 


for  another  128  kilobytes  per  patient.  Different  media  may  be  chosen  to  store 
patient  data. 
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3.4.1  Video  Tape 

Standard  video  tape  cartridges  have  the  advantage  of  high  data  capacity.  A 
single  video  tape  could  store  a  single  patient’s  data  file  in  a  fraction  of  a  second 
of  video  tape.  However,  the  video  tape  is  bulky  to  store  and  suffers  age  related 
degradation.  Furthermore,  data  requires  sequential  access. 

3.4.2  Diskette 

Currently,  storage  on  standard  computer  ‘floppy’  disks  is  feasible.  The  max¬ 
imum  disk  capacity  is  currently  limited  to  1.44  megabytes  per  disk.  Thus  a 
single  patient’s  data  file  would  require  6  diskettes  for  storage.  Although  these 
disks  are  small,  6  disks  per  patient  would  become  cost  and  storage  space  pro¬ 
hibitive.  A  more  realistic  alternative  would  be  a  hard  disk  drive  for  patient 
files.  Disk  drives  with  capacities  in  excess  of  a  gigabyte  are  now  common. 

3.4.3  Optical  Disk  Storage 

Optical  disk  technology  is  now  available  for  image  storage.  Re-writeable  optical 
disks  are  available  in  400  megabyte  capacities.  Disks  cost  approximately  200 
dollars  each.  Optical  disk  drives  for  image  or  data  storage  cost  approximately 
5,000  dollars.  The  data  transfer  speed  for  optical  disks  are  200  kilobytes  per 
second  [40]. 


3.4.4  Digital  Audio  Tape 

Digital  audio  tape  (DAT)  technology  shows  promise  for  image  storage.  These 
tape  cartridges  are  approximately  three  inches  long  by  two  inches  wide.  Car¬ 
tridges  are  now  available  with  capacities  to  1.2  gigabytes  at  a  cost  of  30  dollars. 
Although  the  tape  is  accessed  sequentially,  a  file  can  be  found  in  less  than  20 
seconds.  The  data  transfer  rate  for  the  DAT  is  192  kilobytes  per  second.  Reed- 
Solomon  error  correcting  techniques  are  used  for  data  reliability  [40].  Digital 
audio  tape  drives  are  now  available  as  options  on  personal  computers  for  under 
400  dollars. 

3.4.5  Storage  Medium  of  Choice 

The  storage  medium  of  choice  for  this  subsystem  is  the  standard  hard  disk  due 
to  its  ready  availability.  However,  the  DAT  system  is  also  desired  as  an  archival 
system  for  clinics  with  a  large  patient  base. 

3.5  Retinal  Observation  Software 

Retinal  observation  software  was  written  to  support  the  need  for  image  acqui¬ 
sition,  storage,  and  enhancement.  A  structure  chart  of  the  RETINA  software 
was  provided  earlier  in  this  chapter.  This  section  provides  a  brief  review  of  sim¬ 
ilar  software  that  is  available  and  highlights  the  capabilities  of  the  RETINA 
software. 
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3.5.1  Previous  Work 

There  are  many  software  packages  that  have  been  developed  to  image  the  retina 
and  its  specific  features.  Also  packages  have  been  developed  to  provide  spe¬ 
cialized  functions  such  as  measure  cup  to  disc  ratios  and  create  a  mosaic  map 
of  the  retina.  These  packages  are  too  numerous  to  separately  detail  here.  The 
interested  reader  is  referred  to  the  bibliography  [41,  42,  43.  44,  45,  46,  47]. 
Some  of  these  ideas  were  adapted  for  use  in  the  RETINA  software. 

3.5.2  RETINA  Software 

Program  RETINA  (Retinal  Tracking  and  Image  Analyzer)  is  a  user  friendly, 
menu  driven  program  developed  to  image  and  track  the  retina.  The  program 
is  written  in  Microsoft  C  version  6.0.  It  requires  approximately  4  megabytes  of 
random  access  memory  and  a  runtime  stack  size  of  24.000  bytes.  The  program 
requires  a  Matrox  PIP-1024  frame  grabber  with  its  accompanying  software 
library  PIP-EZ  and  a  Data  Translation  DT-2801 A  data  acquisition  system  and 
its  accompanying  library  PC  LAB. 

Top  down  software  design  techniques  were  used  in  software  development. 
Also,  the  Air  Force’s  Reliability  and  Maintainability  2000  (R&M  2000)  software 
design  techniques  were  applied  in  development  [48].  The  program  is  completely 
modular  for  ease  of  modification  and  update. 

The  program  is  centered  about  a  main  menu  from  which  the  user  can 
call  specific  submenus.  These  main  menu  selections  include:  image  utilities, 
advanced  imaging  functions,  edge  detection  algorithms,  image  statistical  func¬ 
tions,  template  and  lesion  data  base  building  functions,  and  the  actual  tracking 
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algorithm.  There  are  also  numerous  support  functions  to  interface  the  software 
to  the  Hardware/Software  Interface,  a  linked  list  abstract  data  type,  and  func¬ 
tions  to  time  code  execution.  Specific  functions  will  be  detailed  throughout  this 
document.  Functions  identified  with  an  asterisk  (*)  on  the  structure  chart  are 
library  functions  provided  with  the  PIP-EZ  or  the  PCLAB  software  libraries. 


Chapter  4 


The  Retinal  Tracking  Subsystem 

4.1  Objective 

The  purpose  of  the  Retinal  Tracking  Subsystem  is  to  measure  and  compensate 
for  eye  movements  during  laser  treatment  for  diabetic  retinopathy  and  other 
retinal  diseases  treatable  by  photocoagulation.  The  system  designed  and  im¬ 
plemented  in  this  project  uses  digital  video  picture  registration  to  accomplish 
this  task.  This  chapter  reviews  the  theoretical  basis  of  picture  registration 
followed  by  a  discussion  of  the  ideal  tracking  algorithm.  The  ideal  algorithm 
will  be  used  as  a  benchmark  to  measure  the  effectiveness  of  existing  tracking 
algorithms.  This  is  followed  by  a  short  overview  of  the  tracking  algorithm.  The 
remainder  of  the  chapter  details  the  tracking  algorithm.  Testing  and  timing  of 
the  tracking  algorithm  is  detailed  in  a  separate  chapter. 

4.1.1  Theoretical  Basis 

Conceptually,  digital  video  picture  registration  uses  digitized  video  images  to 
determine  the  amount  of  object  movement  that  has  occurred.  A  reference 
image  of  the  object  is  first  obtained.  Subsequent  images  of  the  object  are 
compared  to  the  reference  image  to  determine  the  amount  of  object  movement. 
If  information  on  object  movement  is  provided  to  a  system  for  adjustment, 
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object  tracking  has  occurred. 

The  object  in  our  discussion  is  the  retina.  Ghaffari  has  conceptualized 
the  steps  required  to  track  the  retina  [49]. 

1.  Store  a  retinal  video  image  as  the  reference  image. 

2.  Find  the  position  of  the  best  match  between  the  reference  image  and  the 
present  image. 

3.  Calculate  the  amount  of  movement. 

4.  Update  the  system  with  the  amount  of  movement  that  has  occurred. 

5.  Repeat  steps  2  through  4. 

Intuitively,  the  amount  of  time  required  to  accomplish  step  2  is  directly 
related  to  the  size  of  the  search  area  and  the  complexity  of  the  algorithm 
to  match  the  images.  Furthermore,  step  2  is  complicated  by  complex  retinal 
movements  involving  rotation  about  3  axes  of  rotation.  These  axes,  called  the 
X,  Y,  and  Z  axis  of  Fick,  rotate  about  the  eye’s  center  of  rotation.  Listing’s 
plane  passes  through  the  center  of  eye  rotation  and  contains  the  X  and  Z  axes 
of  Fick.  Video  imaging  describes  movement  relative  to  the  imaging  plane  as 
translation,  rotation,  and  scale.  Translation  is  object  displacement  along  the 
image  abscissa,  ordinate,  or  both.  Rotation  is  an  angular  displacement  about 
a  specific  axis.  Scale  is  an  object’s  increase  or  decrease  relative  to  a  fixed 
ratio  within  an  image.  These  terms  are  illustrated  in  Figure  4.i.  Methods  to 
measure,  minimize,  and  compensate  for  these  movements  are  provided  later  in 
this  document. 


48 


Y 


frame  grabber  image 
plane 


Scale 


Figure  4.1:  Top:  the  X,  Y,  and  Z  axes  of  Fick  rotation  as  related  to  the  image 
plane  frame  of  reference  [adapted  from  [50]].  Bottom,  left  to  right:  reference 
image,  translation,  rotation,  and  scale  movements  related  to  a  retinal  image. 
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4.1.2  The  Ideal  Tracking  Algorithm 

To  design  an  effective  tracking  algorithm  a  description  of  an  ideal  algorithm 
is  useful  as  a  guide  for  development,  as  a  benchmark  for  evaluation,  and  as  a 
planning  aid  for  future  algorithm  improvements.  This  section  provides  a  list  of 
requirements  for  an  ideal  retinal  tracking  system.  Such  a  system  should  have 
the  following  capabilities: 

1.  Track  all  retinal  movements  and  provide  the  necessary  corrective  signals 
to  compensate  for  the  movement. 

2.  Have  zero  response  time. 

3.  Be  impervious  to  changes  in  retinal  illumination. 

4.  Track  effectively  in  all  retinal  fields  of  view. 

5.  Track  effectively  in  pathological  degraded  conditions  such  as  cataracts 
and  vitreous  opacities. 

6.  Protect  the  critical  vision  anatomy  (the  fovea  and  optic  disk)  and  the 
retinal  vessels  from  laser  irradiation. 

7.  Respond  to  patient  input  (panic,  blinks,  etc.). 

8.  Have  control  over  the  laser  positioning  system  and  the  laser  shutter. 

9.  Track  in  the  absence  of  visible  retinal  features. 


10.  Incorporate  or  not  be  affected  by  newly  developed  retinal  lesions. 
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Figure  4.2:  The  RETINA  Tracking  Algorithm. 


4.1.3  Overview 

The  flow  chart  for  the  tracking  algorithm  designed  for  this  project  is  provided 
in  Figure  4.2.  This  algorithm  has  been  given  the  name  Retinal  Tracking  and 
Image  Analyzer  (RETINA). 

The  tracking  algorithm  must  attend  to  many  tasks.  These  include: 
identifying  the  patient,  retrieving  the  patient  file  from  the  data  base,  loading 
data  for  the  retinal  field  of  view  for  treatment  from  the  patient’s  file,  controlling 
the  fixation  device  to  minimize  eye  movement,  establishing  tracker  initial  lock 
on  the  retina,  performing  tracking,  and  responding  to  different  contingencies 
such  as  loss  of  tracker  lock,  patient  blink,  and  patient  panic.  Furthermore, 
the  tracking  algorithm  must  respond  to  inputs  from  the  Reflectance  Based 
Feedback  Control  System  and  provide  inputs  to  the  Laser  Pointing  Subsystem. 
Each  one  of  these  tasks  will  now  be  discussed. 


4.2  Lesion  Data  Base 


The  Lesion  Data  Base  (LDB)  is  a  data  bank  that  contains  the  information  for 
lesion  placement  and  size  for  the  treatment  of  diabetic  retinopathy  or  retinal 
breaks  and  tears.  The  construction  of  a  data  base  for  treatment  of  macular 
degeneration  is  not  presented  since  treatment  protocol  for  this  disease  is  similar 
to  diabetic  retinopathy  treatment. 

Development  of  an  LDB  is  complicated  by  the  following  factors: 

1.  Standard  fundus  cameras  for  imaging  the  retina  have  a  maximum  field 
of  view  of  60  degrees  [51].  The  fundus  camera  used  in  this  study  had  a 
maximum  field  of  view  of  50  degrees.  This  limitation  requires  a  separate 
data  base  for  each  of  the  retinal  field  of  views  requiring  treatment.  These 
field  of  view  data  bases  must  be  independently  retrieveable  and  be  de- 
conflicted  from  adjoining  field  of  view  data  bases.  Deconfliction  prevents 
multiple  treatment  of  a  given  retinal  area.  Furthermore,  it  would  be  very 
convenient  if  all  data  for  a  given  patient  could  be  stored  in  a  single  file. 

2.  Critical  vision  anatomy  should  be  spared  from  laser  irradiation.  This 
anatomy  includes  the  optic  disk,  the  fovea  and  macula,  and  the  retinal 
vessels  [1,  21]. 

3.  Minimal  user  input  should  be  necessary  for  effective  use  of  the  clinician’s 
time. 

The  LDB  methodology  developed  for  this  research  minimizes  these  com¬ 
plications.  The  following  description  details  the  multiple  stages  of  image  pro¬ 
cessing  required  to  develop  a  complete  patient  LDB. 
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Figure  4.3:  The  field  of  view  numbering  system. 

The  first  step  in  developing  an  LDB  for  a  given  retinal  field  of  view  is  to 
identify  the  patient  and  the  field  of  view.  The  patient  file  is  identified  by  the 
patient’s  initials  followed  by  an  underscore  and  the  patient’s  last  four  digits 
of  their  social  security  number  (for  example:  sfb_8253).  This  file  designator 
provides  a  short,  descriptive,  unique  file  label.  Fields  of  view  are  identified  using 
a  single  letter  (T  or  ‘r’)  to  identify  the  eye  and  integers  (1  to  16)  to  identify 
the  field  of  view  for  treatment.  Fields  of  view  are  numbered  as  illustrated  in 
Figure  4.3.  Only  16  fields  of  view  are  currently  programmed  for  each  eye.  This 
is  a  sufficient  number  for  proof  of  design.  The" design  can  be  easily  extended  to 
any  number  of  field  of  views. 

The  individual  field  of  view  data  bases  are  kept  separate  within  a  pa¬ 
tient’s  file  using  a  hashing  function.  The  hashing  function  provides  unique 
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Patient  Data  File 

File  Designator:  fml  OOOl 

Left  Eye  Template  Data 


Right  Eye  Template  Data 


Left  Eye  Lesion  Data 


Right  Eye  Lesion  Data 


Figure  4.4:  The  patient  file. 

header  and  trailer  data  words  which  signal  the  beginning  and  end  of  a  .".eld  of 
view’s  lesion  data  [52].  A  similar  hashing  function  is  used  to  store  the  field  of 
view  template  data  in  the  patient’s  file.  A  sample  diagram  of  the  patient's  file 
is  provided  in  Figure  4.4. 

After  the  patient  file  has  been  specified  and  opened  for  writing  and  the 
appropriate  field  of  view  hashing  header  has  been  generated  the  LDB  building 
function  prompts  the  user  for  the  location  of  the  optic  disk  on  the  image.  A 
cursor  is  provided  for  the  user  to  align  with  the  center  of  the  optic  disk  (if 
required).  The  function  then  initiates  12  steps  of  image  processing  to  build 
the  LDB.  The  steps  include  functions  to  enhance  the  retinal  vessels  via  edge 
detection  followed  by  filtering  steps  to  remove  noise.  The  retinal  vessel  map 
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Template  Response  -  (Sum  1  -  Sum  2)  +  (Sum  4  -  Sum  3) 

Figure  4.5:  Modified  Markow  vessel  enhancement  templates 

generated  is  then  converted  to  a  binary  image.  The  critical  vision  anatomy 
is  then  masked  from  lesion  placement.  Lesion  coordinates  are  then  plotted 
and  stored  in  the  patient  file.  The  user  may  then  test  the  LDB  for  a  visual 
demonstration  of  lesion  placement.  Additional  details  on  the  LDB  process  are 
now  provided. 

The  reference  retinal  image  is  first  scanned  with  horizontal  and  vertical 
templates  of  five  pixel  width  to  emphasize  the  vessel  network.  This  template 
width  equates  to  a  vertical  blood  vessel  width  of  175  microns  and  a  horizontal 
vessel  width  of  145  microns  (for  development  system  configuration).  Additional 
vessel  widths  were  tested  but  did  not  significantly  add  to  the  enhancement 
provided  by  the  5  pixel  width  template.  Figure  4.5  provides  a  diagram  of  this 
modified  Markow  template  operation.  A  detailed  discussion  of  the  Markow 
template  is  contained  later  in  this  chapter. 

The  results  of  the  horizontal  and  vertical  vessel  enhancement  steps  are 
then  combined  after  median  filtering.  A  median  filter  provides  image  smoothing 


but  does  not  blur  edges  [34].  Horizontal  and  vertical  vessels  enhancement  steps 
are  combined  by  choosing  the  highest  template  response  from  the  two  steps  at 
a  given  image  coordinate. 

The  surrounding  image  edge  effects  are  then  ‘trimmed  off’  since  they 
are  an  artifact  from  the  fundus  camera.  The  binary  vessel  map  is  then  formed 
by  allowing  the  user  to  determine  the  thresholds  for  the  binary  function.  The 
result  is  a  white  (255)  vessel  map  on  a  black  (0)  background.  The  user  is 
allowed  to  iterate  as  required  to  obtain  the  desired  vessel  map.  To  remove 
spot  noise  from  the  binary  vessel  map,  each  white  (255)  pixel  is  examined  for 
connectivity.  If  a  white  pixel  is  found  to  have  an  adjoining  white  neighbor  it  is 
considered  to  be  an  information  pixel  otherwise  it  is  considered  spot  noise.  The 
spot  noise  identified  pixels  are  immediately  mapped  to  black  (0).  This  results 
in  a  significant  decrease  in  image  spot  noise. 

After  the  binary  vessel  map  is  complete,  the  critical  anatomy  is  pro¬ 
tected  by  masking  the  critical  area  of  the  fovea  and  optic  disk.  The  area 
surrounding  the  tracking  templates  is  also  protected  to  prevent  irradiation  and 
modification  of  these  tracking  landmarks.  Lesion  coordinates  are  then  gener¬ 
ated  and  stored  in  the  patient’s  file. 

Lesions  are  placed  as  prescribed  for  the  treatment  of  diabetic  retinopa¬ 
thy  or  retinal  breaks  or  tears.  The  individual  field  of  view  data  bases  do  not 
overlap.  The  LDB  building  process  is  summarized  in  Figure  4.6.  The  optic 
disk  is  visible  in  fields  of  view  one  through  seven.  Therefore,  the  optic  disk 
and  other  vision  critical  anatomy  must  be  identified  and  protected  from  lesion 
placement.  Results  are  provided  in  Figures  4.7  through  4.11. 
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Figure  4.6:  Lesion  Data  Base  building 
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Figure  4.9:  Left  to  right:  combine  horizontal  and  vertical  enhancements,  re 
move  edge  effects,  binary  image 


Figure  4.10:  Left  to  right:  examine  neighborhood  for  connectivity,  protect 
critical  anatomy,  plot  coordinates 
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Figure  4.11:  The  final  result 


4.2.1  Panretinal  Photocoagulation  Treatment  for  Diabetic  Retinopa¬ 
thy 

As  discussed  earlier  in  this  document  treatment  for  diabetic  retinopathy  con¬ 
sists  of  placing  laser  lesions  on  the  retina  in  a  circular  pattern  about  the  region 
of  critical  vision  anatomy.  The  lesion  pattern  consists  of  two  rings  of  200  mi¬ 
cron  lesions  spaced  300  microns  from  center  to  center.  The  outer  rings  are  500 
micron  lesions  spaced  750  microns  center  to  center.  The  lesions  are  no  closer 
than  one-half  of  a  lesion  diameter  from  the  nearest  adjoining  retinal  vessel. 

4.2.2  Treatment  for  Retinal  Breaks  or  Tears 

The  treatment  for  retinal  breaks  or  tears  consists  of  placing  two  rings  of  200 
micron  lesions  about  the  damaged  area.  These  lesions  are  placed  200  microns 
from  center  to  center  to  form  a  continuous  ring.  The  spacing  between  adjacent 
rings  is  300  microns  center  to  center  [1,  53] .  A  separate  imaging  process  is 
required  to  develop  a  LDB  for  treatment  of  retinal  breaks  or  tears.  This  process 
must  have  the  capability  and  flexibility  to  place  lesions  around  retinal  tears  of 
irregular  size  and  shape. 

A  flow  chart  for  the  algorithm  developed  to  build  a  retinal  break  or  tear 
data  base  is  provided  in  Figure  4.12. 

The  initialization  steps  are  similar  *o  those  for  building  a  diabetic  retinopa¬ 
thy  treatment  data  base.  These  steps  identify  the  field  of  view  for  treatment 
and  generate  hash  markers  for  data  storage  in  the  patient's  file.  The  bound¬ 
aries  of  the  treatment  area  are  also  defined  to  reduce  the  amount  of  processing 
time  required  to  build  the  break  or  tear  data  base. 


63 


Build  Retinal  Tear/Break  Data  Base 


Figure  4.12:  Retinal  break  or  tear  data  base  building. 
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The  treatment  data  base  is  formed  by  drawing  a  white  (255)  ‘stick  figure' 
over  the  retinal  tear.  This  figure  may  be  any  shape.  However,  the  figure  must 
be  continuous.  After  the  user  draws  the  stick  figure  using  the  cursor  control,  a 
line  is  drawn  from  the  stick  figure  to  the  edge  of  the  detached  area.  This  line 
defines  the  extent  of  the  treatment  area.  The  algorithm  then  masks  the  area 
over  the  stick  figure.  This  is  done  by  detecting  the  presence  of  the  stick  figure 
by  searching  for  white  pixels  (255)  in  the  treatment  area.  If  a  white  pixel  is 
found  a  circular  mask  is  plotted  at  the  white  pixel  coordinate  with  a  radius 
equal  to  the  extent  of  the  the  damaged  area.  This  process  continues  over  the 
entire  stick  figure.  The  result  is  a  masked  area  about  the  detached  area.  A 
single  pixel  boundary  is  then  plotted  about  the  mask  area  by  detecting  the 
mask’s  edge. 

Data  base  development  continues  by  searching  for  a  white  boundary 
pixel  within  the  defined  treatment  area.  When  a  boundary  pixel  is  found 
it  is  designated  as  the  first  boundary  pixel.  The  boundary  is  then  traced  by 
examining  the  neighborhood  of  pixels  about  the  first  boundary  pixel  for  another 
boundary  pixel.  If  another  white  boundary  pixel  is  found  the  new  .direction 
flag  is  set.  This  process  continues  around  the  boundary  until  the  boundary 
tracing  operation  returns  to  the  first  boundary  pixel.  At  every  fifth  pixel  (ss 
200  microns)  around  the  boundary  a  lesion  coordinate  is  plotted.  The  entire 
process  is  repeated  with  a  larger  mask  to  provide  the  second  ring  of  200  micron 
pixels. 

If  during  the  boundary  tracing  operation  another  boundary  pixel  is  not 
found,  the  function  attempt -boundary  .connect  tries  to  re-establish  boundary 
contact  about  the  last  known  boundary  pixel.  Up  to  eight  attempts  are  allowed 


in  all  directions  from  the  last  known  pixel  to  re-establish  the  boundary  tracing 
operation.  The  development  of  the  retinal  break  or  tear  data  base  is  illustrated 
in  Figures  4.13  through4.15. 

4.3  Template  Building 

The  next  step  after  building  the  LDB  is  to  construct  a  template  for  tracking 
retinal  movement.  In  the  actual  software  steps,  template  building  precedes 
LDB  building.  This  is  required  to  protect  the  template  tracking  landmarks 
from  lesion  modification  or  destruction.  This  section  details  the  construction 
of  the  tracking  template.  Template  theory  is  reviewed  followed  by  a  review  of 
previous  work  in  this  area.  A  detailed  description  of  template  selection  follows. 

4.3.1  Template  Theory 

Digital  picture  registration  uses  the  technique  of  template  matching  [54].  Bovik 
provides  a  good  review  of  the  template  matching  theory  [55].  The  discussion 
below  is  based  on  his  coverage  of  the  topic. 

Template  matching  is  based  on  having  a  reference  image  of  an  object. 
For  this  discussion,  this  reference  image  will  be  designated  as  I.  In  the  retinal 
tracker,  I  will  be  a  512  x  512  digitized  image  of  the  retina  with  256  different 
gray  levels  of  contrast.  Multiple  images  of  this  type  are  required  to  map  the 
retina.  Only  a  single  field  of  view  will  be  considered  at  a  time  with  the  retinal 
tracker.  Within  the  reference  image  I  are  distinct  features.  In  the  retina  these 
features  are  the  retinal  vessels  and  photocoagulation  lesions.  The  movement  of 
these  features  between  the  reference  image  and  subsequent  images  are  used  to 
determine  overall  retinal  movement. 
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Figure  4.13:  Left  to  right:  simulated  retina!  tear,  treatment  area  definition, 
drawing  the  ‘stick  figure’ 


68 


Figure  4.15:  Left  to  right:  the  outer  mask,  outer  boundary,  final  result 
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Figure  4.16:  The  image  I  and  the  template  T. 


A  window  may  be  constructed  that  contains  one  or  more  of  the  distinct 
vessel  features.  This  window  is  a  sub-image  of  I.  This  window  designated  as 
the  template  T  is  a  sub-image  of  I.  The  m  x  n  dimension  of  T  is  much  less 
than  the  dimension  of  I.  The  relationship  between  I  and  T  is  diagrammed  in 
Figure  4.16. 

The  template  T  will  overlay  the  image  I.  The  set  of  pixels  covered  by 
the  template  T  when  it  overlays  I  at  coordinate  (i.j)  is  given  by: 

T  •  I(i,  j)  =  I(i  +  m,i  +  n);  (m.n)  e  T  (4.1) 

There  are  several  different  ‘match’  measures  that  may  be  defined  to 
determine  the  best  match  between  template  T  and  T  I(i,y).  When  T  is 
placed  at  different  coordinates  on  I  and  a  measure  is  applied  to  each  of  these 
coordinates  where  T  can  be  placed  a  match  image  J  results.  The  match  image 


J  will  have  a  maximum  (or  minimum  depending  on  the  measure)  at  positions 
where  a  good  match  occurs. 
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Some  measures  are  more  appropriately  designated  as  mismatch  mea¬ 
sures  since  they  provide  a  small  numerical  value  when  gray  levels  in  T  and 
T  •  I (i,j)  are  similar.  Different  methods  of  measuring  mismatch  include  maxi¬ 
mum  absolute  error,  mean  absolute  error,  and  mean-square  error.  The  mean- 
square  error  technique  can  be  dissected  and  modified  to  form  a  good  match 
measure. 

The  mean-square  error  mismatch  measure  is  given  by: 


MSE  T  -  I(i,  j),T  =  53  53  [I(*  +  m,  j  +  n)  ~  T(m,n)]2  (4.2) 

m,n€:T 

This  measure  can  be  decomposed  into  three  separate  parts  including:  a 
total  template  energy  term  Ej  which  is  constant  with  respect  to  (i,j),  a  local 
image  energy  term  £x  i(i,j)  at  (bj)  which  provides  no  information  on  a  specific 
T  and  a  cross-correlation  term  of  I  and  T. 

The  cross-correlation  term  is  given  by: 

Ci,t(*\j)  =  53  13  *(*  +  m>7  +  rc)T(m,n)  (4.3) 

m,n£  T 

The  mean-square  error  with  this  terminology  may  be  written  as: 


MSET  ■  I (i,  j),  T  =  Ej  +  £t-I(«,j)  -  2Cx  t(i,  j) 


(4.4) 


Since  the  mean-square  error  is  quantitatively  small  when  there  is  a  good 
match,  must  increase  at  this  match  location.  Recall,  Ej  and  Ej;.i(i,j) 

are  not  affected  by  match  conditions. 

The  Schwarz  Inequality  provides  an  upper  bound  on  the  size  of  the 
cross-correlation  Cxt(*i  j)- 

Cl.T (i,j)  <  y/Er-lihj)  ■  £t  (4-5) 

To  determine  the  relative  goodness  of  the  match,  the  cross-correlation 
must  be  compared  to  this  upper  bound.  This  can  be  accomplished  by  defining 
the  normalized  cross-correlation  CjT(i,j)  as: 

Cl.T (i,j)  =  CitT(i,j)/y/ET.i(i,j )  ■  Et  (4-6) 

Then: 

Cit (i,j)  <  CiT(iJ)  <  1  for  every  (i,j)  (4.7) 

The  normalized  cross-correlation  may  then  be  thresholded  by: 

A =  I  *f  Cf,T (iyj)  <  1  (4-8) 

to  identify  a  point  having  a  good  match  [55].  The  threshold  must  dis¬ 
tinguish  between  the  single  correct  match  and  other  potential  match  locations. 

Once  a  match  is  determined,  movement  may  be  calculated.  The  differ¬ 
ence  in  position  between  the  template  position  on  the  reference  image  and 
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the  template  position  on  subsequent  images  provides  a  quantitative  measure 
of  retinal  movement.  This  information  may  be  provided  as  feedback  to  reorient 
the  laser  positioning  mirrors  to  an  updated  position.  The  overall  effect  is  to 
maintain  the  position  of  the  laser  on  the  same  lesion  location.  The  Retinal 
Tracking  Subsystem  must  be  able  to  resolve  within  the  laser  spot  size  to  com¬ 
pensate  for  movement  during  irradiation.  The  response  time  of  this  system 
must  be  faster  than  typical  eye  movements  -  maximum  velocity  of  up  to  800 
degrees  per  second  and  maximum  acceleration  of  40,000  degrees  per  second  [2]. 

4.4  The  Tracking  Algorithm 

Tracking  the  movement  of  the  human  retina  is  not  a  new  idea.  As  early  as 
1968  West  described  a  system  for  laser  position  control  near  the  fovea  [56]. 
According  to  Timberlake  the  system  was  probably  not  built  [57].  Kelly  and 
Crane  completed  a  detailed  study  in  1968  for  the  National  Aeronautics  and 
Space  Administration  (NASA)  for  a  fundus  tracker.  Their  method  involved 
“projecting  a  scanning  pattern  onto  the  retina,  and  detecting  the  translational 
and  rotational  movements  of  the  reflected  pattern  by  means  of  a  certain  type 
of  high-speed  correlation  processing  of  the  video  signal.”  They  visualized  the 
real  system  operating  at  a  rate  from  200  to  1000  updates  per  second.  Due 
to  equipment  limitations  they  tested  their  apparatus  with  video  updates  every 
few  seconds  [58]. 

Since  that  initial  work  many  researchers  have  attempted  to  track  the 
movement  of  the  human  retina  using  various  technologies.  This  section  begins 
with  a  review  of  previous  tracking  technologies  and  algorithms  and  concludes 
with  a  detailed  description  of  the  tr?.cking  algorithm  developed  for  this  project. 
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4.4.1  Previous  Work 

Double- Purkinje  Image  Eyetracker 

A  decade  after  the  work  of  Kelly  and  Crane,  Crane  and  Steele  developed  an  op¬ 
tical  system  to  track  the  movement  of  the  retina  using  a  double- Purkinje  image 
method.  This  system,  called  the  Double-Purkinje  Image  eyetracker,  detected 
the  reflection  of  an  near  infrared  (0.93  microns)  beam  projected  into  the  eye. 
The  beam  was  reflected  from  the  four  optical  surfaces  of  the  eye:  the  anterior 
and  posterior  surfaces  of  the  cornea  and  the  anterior  and  posterior  surfaces  of 
the  crytalline  lens.  These  four  reflected  images  are  called  the  Purkinje-Sanson 
images  I,  II,  III,  and  IV  [16].  The  tracker  monitored  the  movement  of  the  I 
and  IV  images  to  derive  information  on  retinal  rotation  and  translation.  This 
system  required  a  dental-impression  bite  board  and  a  2-point  forehead  rest  to 
stabilize  the  patient’s  head  against  movement  [59,  60].  Recent  improvements 
in  this  design  have  resulted  in  tracking  capability  to  a  maximum  of  100  degrees 
per  second  with  a  response  time  of  0.13  seconds  [61]. 

Feature-Based  Registration  of  Retinal  Images 

Several  researchers  have  used  the  natural  features  of  the  retina,  most  notably 
the  optic  disk  and  the  retinal  vessels,  to  track  retinal  eye  movement.  Barnea 
and  Silverman  developed  a  sequential  similarity  detection  (SSD)  technique  to 
measure  eye  movement.  This  technique  reduced  the  computation  time  by  a 
factor  of  100  over  then  current  cross- correlation  techniques.  The  SSD  technique 
uses  the  sum  of  the  absolute  values  of  the  differences  (SAVD)  to  measure  the 
similarity  of  a  reference  template  to  an  image.  The  similarity  measure  is  defined 
by: 
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J  K 

SAVD(m,  n)  =  (4.9) 

j= 1  k=  1 

where: 

•  F\(j,  k)  is  a  J  x  K  pixel  template. 

•  Fi(j,  k)  is  a  M  x  N  pixel  search  area. 

This  technique  provided  normalization  to  account  for  illumination  dif¬ 
ferences  between  the  template  and  the  search  area  and  implemented  a  thresh¬ 
olding  technique  to  reduce  computation  time.  The  thresholding  technique  was 
implemented  by  first  examining  similarity  at  only  randomly  selected  points 
within  the  template.  During  the  examination  of  similarity  a  running  total  of 
accumulated  error  was  compared  to  the  pre-established  threshold.  When  the 
threshold  was  exceeded  the  sirr  ilarity  calculation  for  the  template  at  the  present 
search  area  location  was  aborted.  Additional  points  of  the  template  were  com¬ 
pared  with  the  search  area  if  the  threshold  was  not  exceeded.  The  coordinate 
of  the  search  area  that  allowed  the  most  similarity  point  calculations  prior  'o 
exceeding  the  threshold  was  declared  the  match  [62]. 

Peli  et  al.  [63]  improved  on  the  Barnea  and  Silverman  algorithm  by  se¬ 
lecting  template  points  for  calculating  SAVD  from  the  positions  of  the  vessels  in 
the  template.  This  modification  reduced  the  computation  time  of  the  similar¬ 
ity  measure.  To  further  reduce  processing  time  a  two-stage  template  matching 
algorithm  was  also  implemented.  The  two-stage  process  used  a  coarse  search 
to  determine  the  most  likely  area  of  the  match.  This  was  accomplished  by  skip¬ 
ping  over  rows  and  columns  in  the  search  area.  The  coarse  search  was  followed 
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by  a  fine  search  about  the  coarse  search  match  point.  Although  this  technique 
reduced  processing  time,  Peli  noted  the  following:  “The  most  important  dif¬ 
ficulty  encountered  with  this  coarse-fine  approach  is  that  coarse  subsampling 
may  actually  skip  the  important  points  in  the  search  area,  resulting  in  erro¬ 
neous  determination  in  the  first  stage.  This  is  especially  true  for  a  pattern  with 
small,  sharp  details  such  as  that  seen  in  retinal  vessels”. 

Acousto-Optical  Cells 

G'naffari  investigated  using  the  optical  image  correlation  technique  for  retinal 
tracking.  In  this  technique,  acousto-optical  (A-O)  cells  performed  the  fine 
image  matching  operation.  The  cells  “convert  the  real-time  one-dimensional 
video  information  into  a  spatial  light  intensity  controller  system."1  Specifically, 
a  two-dimensional  template  was  generated  to  store  a  reference  image.  This 
image  was  then  correlated  to  the  real-time  video  frames.  The  results  of  this 
correlation  was  a  new  light  distribution  with  a  relative  maximum  located  at 
the  center  of  the  best  match.  Retinal  movement  was  calculated  by  measuring 
the  movement  of  the  relative  correlation  image  maximum  from  one  frame  to 
the  next.  Ghaffari  tested  this  system  by  tracking  several  different  character 
symbols  and  simulated  biood  vessels.  He  reported  “a  ±  2.0%  linearity  in  the 
horizontal  and  ±  1.8%  linearity  in  the  vertical  directions.  The  tracking  system 
can  handle  speed  and  acceleration  of  656  degrees  per  second  and  19,687  degrees 
per  second2  for  30  frames  per  second  video  rate.  The  accuracy  of  tracking  is 
within  ±5  pixels  in  a  275  pixel  square  area” [49]. 
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Scanning  Laser  Ophthalmology 

Several  researchers  have  used  the  scanning  laser  ophthalmoscope  to  track  the 
movement  of  the  retina.  The  algorithm  developed  calculates  the  cross- correlation 
between  preselected  binary  templates  with  binary  retinal  images.  The  binary 
images  are  coded  with  +1  =  white  and  -1  =  black.  This  coding  scheme  allows 
the  calculation  of  the  Hamming  distance  as  a  mismatch  measure  between  the 
template  and  the  image  under  search.  The  researchers  indicate  that  computa¬ 
tion  time  may  be  reduced  by  performing  the  cross-correlation  in  the  frequency 
domain  using  standard  Fast  Fourier  Transform  (FFT)  techniques.  This  reduces 
the  computation  time  to  2  seconds  for  a  512  x  512  pixel  image.  The  tracker  is 
able  to  compute  the  match  position  within  2  pixels  of  a  512  x  512  pixel  image 
[64,  65], 

Markow's  Blood  Vessel  Tracking 

Markow  studied  the  digital  retinal  tracking  problem  in  some  detail.  After  re¬ 
viewing  many  different  tracking  techniques  he  concluded  that  they  were  com¬ 
putationally  exorbinant.  However,  he  borrowed  portions  from  these  techniques 
to  develop  a  tracking  algorithm  using  blood  vessel  templates. 

This  tracking  algorithm  uses  two  one-dimensional  (ID)  blood  vessel  tem¬ 
plates  normally  oriented  to  one  another.  One  template  is  horizontal  and  the 
other  is  vertical.  These  two  templates  are  bounded  by  the  template  window  T 
discussed  earlier  in  this  chapter.  The  ID  templates’  orientation  is  illustrated 
in  Figure  4.17  [2]. 

The  horizontal  ID  template  Ht  and  the  vertical  ID  template  Vt  are 
each  a  set  of  pixels.  Each  of  the  small  squares  within  Figure  4.18  represents 
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Figure  4.17:  One  dimensional  template  orientation  [2] 
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(white  =  255) 
leading  edge 
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Blood  Vessel  Width 


Sum  2  Sum  3 


Sum  4 


Template  Response  =  (Sum  1  -  Sum  2)  +  (Sum  4  -  Sum  3) 


Figure  4.18:  Ihe  ID  template.  For  an  ideal  image  (a  black  (0)  vessel  on  a 
white  (255)  retina)  Sum  1  is  510,  Sum  2  is  0,  Sum  3  is  0,  and  Sum  4  is  510. 
The  overall  template  response  is  1020. 


a  single  pixel.  The  small  black  squares  represent  where  the  edges  of  a  blood 
vessel  would  occur.  To  search  for  a  blood  vessel  of  a  specific  width,  pixels  are 
inserted  between  the  blood  vessel  edge  pixels.  The  number  of  pixels  inserted 
corresponds  to  the  width  of  the  blood  vessel  desired.  Markow  limited  the 
number  of  pixels  for  insertion  from  2  to  9  [2]. 

The  ID  templates  measure  the  template  response  to  determine  where  a 
blood  vessel  s  leading  edge  exists.  To  detect  a  blood  vessel  edge,  the  gray  level 
values  of  the  two  pixels  to  the  right  of  the  small  black  squared  are  summed 
(Sum  2).  Also  the  gray  level  values  of  the  two  pixels  to  the  left  of  the  black 
square  are  summed(Sum  1).  Sum  2  is  then  subtracted  from  Sum  1.  If  a 
leading  edge  exists  at  the  location  of  the  small  black  square,  a  large  number 
will  result.  For  an  ideal  vessel  edge  (a  black  (0)  vessel  on  a  white  (255)  retina) 
the  response  is  510.  The  trailing  edge  of  the  vessel  is  found  in  a  similar  manner. 
The  overall  template  response  is  the  sum  of  the  results  from  the  leading  edge 


and  trailing  edge  templates.  If  a  blood  vessel’s  edges  align  with  the  template, 
a  large  correlation  value  will  result  [2]. 

Templates  Ht  and  Vt  are  initially  selected  by  scanning  the  templates 
of  various  widths  over  the  reference  image.  The  ID  templates  with  the  highest 
correlation  are  retained  to  construct  the  two-dimensional  (2D)  template.  The 
2D  template  consists  of  a  horizontal  and  a  vertical  template  pair  locked  together 
in  a  known  orientation.  The  2D  template  is  used  for  retinal  tracking  [2]. 

In  an  effort  to  speed  up  the  template  searching  process,  Markow  imple¬ 
mented  expansion  techniques.  Rather  than  shifting  the  template  a  pixel  at  a 
time  during  the  search,  it  was  shifted  over  every  fourth  pixel  in  a  5  x  5  pattern. 
This  was  followed  by  a  fine  search  about  the  maximum  point  determined  by 
the  coarse  search.  Reference  Figure  4.19  [2]. 

Markow  initially  tested  this  blood  vessel  template  algorithm  on  a  sim- 
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ulated  blood  vessel.  Qualitatively,  he  demonstrated  the  capability  of  the  al¬ 
gorithm  to  distinguish  between  the  correct  match  and  false  positives.  In  all 
test  cases,  the  correct  match  had  the  highest  value  of  correlation.  The  ratio  of 
the  largest  possible  false  positive  to  the  true  match  decreased  as  the  template 
used  more  ID  templates  and  wider  filter  widths.  The  results  were  inconclusive 
concerning  the  optimum  filter  configuration.  Markow  tested  the  algorithm  on 
a  slowly  rotating  fundus  photograph.  The  tracker  was  able  to  maintain  lock  on 
retinal  images  rotating  at  0  Hz,  1/3  Hz,  1-1/2  Hz  and  5  Hz.  At  higher  frequen¬ 
cies,  the  tracker  could  not  establish  initial  lock.  Markow  also  noted  that  in  all 
test  cases  the  precalculated  expected  value  of  template  correlation  was  much 
higher  than  the  actual  correlation  value  obtained  with  the  tracking  system  [2]. 

Concerns  for  the  Markow  Tracking  Algorithm  The  Markow  Tracking 
Algorithm  was  investigated  in  detail  as  a  smarting  point  for  this  research  effort. 
Markow  did  an  excellent  job  documenting  his  successes  and  areas  of  further 
research  [2].  After  spending  several  months  testing  the  algorithm  and  attempt¬ 
ing  to  optimize  template  selection  for  accuracy  and  speed,  the  following  areas 
of  concern  were  identified: 

•  Markow  employed  a  coarse-fine  search  technique  to  reduce  computation 
time  in  the  searching  process.  As  Peli  [63]  described  “coarse  subsampling 
may  actually  skip  the  important  points  in  the  search  area”.  I  rewrote 
Markow’s  tracking  algorithm  in  C  and  attempted  to  maximize  the  accu¬ 
racy  of  the  algorithm  using  his  template  scheme.  The  algorithm  was  then 
tested  on  actual  human  retinal  images  in  which  movement  was  simulated 
by  picking  different  search  initialization  coordinates.  A  success  rate  of 
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approximately  58%  (the  success  rate  is  defined  as  correct  match  coordi¬ 
nate  found/total  number  of  searches)  was  obtained  at  a  skip  increment 
of  four  pixels.  The  major  cause  of  the  inaccuracy  was  the  coarse  search 
skipping  over  the  correct  coarse  search  pivot  point. 

•  Since  a  coarse-fine  search  technique  was  used  it  was  doubtful  that  a  100% 
accuracy  rate  could  be  obtained. 

•  The  template  Markow  employed  was  not  normalized.  Therefore,  any 
variation  in  retinal  image  intensity  and  nonlinear  imaging  device  response 
could  result  in  an  erroneous  determination  of  the  template  match. 

•  The  template  developed  by  Markow  provided  a  positive  response  over  a 
multi-pixel  region  as  illustrated  in  Figure  4.20.  This  is  a  sound  idea  for 
a  coarse-fine  based  tracking  algorithm;  however,  it  could  result  in  false 
positives. 

•  As  cited  by  Markow,  the  algorithm  could  not  determine  a  loss  of  lock 
condition  [2]. 

•  The  algorithm  did  not  provide  for  the  patient  to  halt  the  tracking  and 
laser  irradiation  process. 

Due  to  these  areas  of  concern,  a  different  method  of  tracking  the  move¬ 
ment  of  blood  vessels  was  required  to  improve  the  accuracy  and  speed  of  the 
algorithm. 

Mayan  Templates  My  colleague,  Dr.  Maya  Jerath,  suggested  using  a  ‘spa¬ 
tially  tuned’  skip  increment  to  increase  the  success  rate  of  the  coarse-fine  search 
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technique.  Her  idea,  which  I’ve  termed  ‘Mayan  Templates’,  uses  a  coarse  search 
skip  increment  equal  to  the  width  of  the  vessel  template  used  in  the  search.  I 
modified  Markow’s  algorithm  to  test  this  concept.  This  modification  allowed 
a  separate  skip  increment  for  both  the  horizontal  and  vertical  directions.  I 
repeated  the  tests  performed  on  the  Markow  algorithm.  The  success  rate  im¬ 
proved  to  approximately  67%  for  all  images  tested  (enhanced  fundus  images, 
flourescein  angiogram  images,  and  scanning  laser  opthalmoscope  images).  The 
success  rate  was  approximately  83%  for  enhanced  fundus  images.  Although, 
this  idea  significantly  improved  the  success  rate.  I  felt  that  an  83%  success 
rate  was  unacceptable  for  equipment  used  to  control  photocoagulation  of  the 
human  retina. 

Branch  Point  Tracking 

Similar  to  the  work  of  Markow  is  the  work  of  Yu  et  al.  This  research  team  has 
developed  a  tracking  algorithm  based  on  detecting  the  movements  of  retinal 
vessel  branch  points.  Their  algorithm  tracks  retinal  movement  by  detecting 
the  location  of  the  optic  disk  via  thresholding  and  three  predetermined  vessel 
branch  points.  The  orientation  of  the  optic  disk  centroid  to  the  three  vessel 
branch  points  forms  the  complete  tracking  template.  The  calculated  displace¬ 
ment  in  retinal  drift  provides  the  driving  signal  to  a  two-dimensional  driving 
stage  which  controls  the  deflection  mirror  in  a  slit  lamp  microscope  [66]. 

The  system  has  been  implemented  on  an  Intel  30386  based  personal 
computer  operating  at  33  MHz.  With  this  computation  power  the  whole  eye¬ 
tracking  process,  including  calculation  of  drift  and  deflection  mirror  adjust¬ 
ment,  requires  1  second.  A  resolution  of  one  pixel  (20  microns)  is  reported  [66]. 


Yu’s  description  did  not  indicate  how  field  of  views  not  containing  the  optic 
disk  are  handled. 
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Dedicated  Fast  Fourier  Transform  (FFT)  Processors 

Considerable  research  is  currently  ongoing  toward  using  dedicated,  high  speed 
reduced  instruction  set  (RISC)  processors  to  perform  FFTs.  These  proces¬ 
sors  are  being  developed  to  provide  tracking  capability  by  performing  template 
matching  in  the  frequency  domain.  The  tracking  is  performed  by  taking  the 
two-dimensional  FFT  of  the  input  image  and  multiplying  it  with  a  frequency 
domain  template  of  the  object  to  be  tracked.  The  result  of  the  multiplication 
is  inversed  transformed.  The  result  is  a  peak  response  at  the  point  of  match 
between  the  image  and  the  template.  Current  processors  have  the  capability 
to  provide  a  tracking  update  15  times  per  second  for  a  256  x  256  input  image 
[67].  This  is  similar  to  Ghaffari’s  optical  correlation  technique. 

4.5  The  Algorithm 

This  section  details  the  tracking  algorithm  designed  and  implemented  for  this 
research.  The  section  begins  with  a  review  of  the  key  assumptions  on  which 
the  tracking  algorithm  is  based.  A  detailed  description  is  then  provided  on  the 
methodology  and  implementation  of  Spatially  Distributed  Normalized  (SDN) 
templates.  The  section  continues  with  a  detailed  description  of  the  tracking 
algorithm  and  its  features.  The  section  concludes  with  a  summary  of  results 
for  the  tracking  algorithm. 


4.5.1  Assumptions  and  Validity  of  Assumptions 
The  tracking  algorithm  is  based  on  the  following  assumptions: 
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Exhaustive  search  required  for  100%  success  rate  To  approach  the 
desired  100%  success  rate,  an  exhaustive  search  of  the  image  search  area  is 
required.  An  exhaustive  search  is  defined  as  a  pixel-by-pixel  scan  of  the  search 
area  to  find  the  match  coordinate.  Tests  conducted  using  various  skip  incre¬ 
ments  indicates  that  the  success  rate  increases  as  the  skip  increment  decreases. 
Even  at  a  skip  increment  of  two  pixels,  errors  caused  by  the  coarse-fine  search 
technique  result. 

Limiting  Eye  Movement  With  Fixation  As  mentioned  in  Chapter  2  of 
this  document,  the  eye  has  a  velocity  of  up  to  800  degrees  per  second  with 
saccadic  eye  movement.  Eye  movement  may  be  minimized  by  having  the  pa¬ 
tient  fixate  on  a  target  with  the  conjugate  eye  while  the  other  eye  is  being 
treated.  Studies  conducted  by  Kosnik  et  al.  [18]  indicate  that  fixation  capabil¬ 
ity  does  not  decrease  significantly  with  age.  This  is  important  since  diabetic 
retinopathy  is  predominant  in  the  elderly  population. 

By  using  a  fixation  device  eye  movements  are  theoretically  relegated  to 
micro-saccades  and  micronystagmus  since  other  types  of  eye  movement  are  for 
acquiring  a  moving  object,  tracking  a  slowly  moving  object,  and  maintaining 
fixation  on  an  object  as  the  observer  moves.  Based  on  this  concept  the  develop¬ 
mental  tracking  algorithm  has  been  designed  around  a  maximum  eye  velocity 
of  50  degrees  per  second.  This  velocity  defines  the  size  of  the  search  area  as 
follows: 


•  The  posterior  nodal  distance  of  the  eye  is  16.7  mm[l6]. 

•  The  velocity  of  retinal  movement  at  50  degrees  per  second  may  be  con¬ 
verted  to  millimeters  per  second  using  an  arc  length  calculation: 

(50  degrees / s)(2n /360  degrees)(l6.7mm)  =  14.6  mm/s  (4.10) 

•  The  standard  frame  rate  of  30  frames/secord  provides  an  image  update 
every  33.3  milliseconds 

•  The  distance  that  a  uoint  on  the  retina  may  move  during  1  frame  is  then: 

(14.6  mm/s)(33.3  ms)  =  .48  mm/ frame  (4.11) 

•  At  a  50  degree  fundus  camera  field  of  view  a  single  pixel  displays  a 
46.4  micron  (horizontal)  by  a  35.3  micron  (vertical)  area.  This  area 
has  been  measured  with  function  measure-displacement.  Function  mea¬ 
sure-displacement  measures  the  displacement  between  two  user  placed 
cursors.  The  function  is  calibrated  to  the  known  dimensions  of  the  optic 
disk. 

•  This  equates  to  a  movement  of  14  pixels  in  any  direction  during  a  frame 
and  a  search  area  of  28  x  28  pixels.  For  safety  the  calculated  area  is 
expanded  to  a  40  x  40  pixel  search  area. 

Minimization  of  rotational  and  scaling  movements  Since  a  fixation 
device  is  used,  rotational  and  scaling  movements  are  minimized.  The  only 
movement  considered  by  the  tracking  algorithm  is  translation.  The  validity  of 
this  assumption  has  been  tested  by  examining  four  different  retinal  movement 
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sequences  in  which  fixation  was  used.  A  detailed  description  of  the  taped 
subjects  is  provided  in  Chapter  8.  The  subjects  will  be  referred  to  as  RCL, 
CSL,  SBR,  and  ICR.  A  scattergram  plotting  the  optic  disk  centroid  for  each 
sequence  has  been  constructed  using  techniques  similar  to  those  of  Kosnik 
et  al.  The  centroid  location  was  plotted  over  a  45  second  period.  Rotation 
was  measured  by  comparing  the  orientation  of  three  landmarks  in  each  frame. 
Sailing  was  measured  by  measuring  the  optic  disk  image  diameter  with  a  caliper 
in  each  frame.  No  rotational  or  scaling  movements  have  been  observed.  This  is 
consistent  with  the  findings  of  De  Castro  et  al  [68].  Also,  no  repeatable  pattern 
has  been  observed  in  the  translational  movement. 

To  further  substantiate  the  two  previous  assumptions  a  40  x  40  pixel 
area  was  ‘snapped’  over  a  3  minute  period  from  subject  SBR’s  retinal  movement 
sequence.  The  results  are  illustrated  in  Figure  4.22.  It  was  clear  from  this  time 
record  that  rotation  and  scaling  is  minimized  with  fixation  and  the  40  x  40 
search  area  calculation  appears  valid. 

Conjugate  Eye  Movement  The  use  of  a  fixation  device  to  minimize  eye 
movement  in  the  conjugate  eye  while  the  other  eye  is  being  treated  depends 
on  the  eyes  moving  as  a  conjugate  pair.  In  most  eye  movements  the  eyes 
move  together.  All  of  the  oculomotor  control  systems  previously  discussed  in 
chapter  3  must  move  the  eyes  together  precisely  to  maintain  binocular  vision. 
Misalignment,  called  heterotropia,  is  due  to  abnormalities  of  muscles  or  nerves 
[69].  The  tracking  system  will  continue  to  operate  correctly  in  the  event  of 
misalignment  if  the  patient  is  able  to  fixate.  If  the  patient  is  not  capable  of 
holding  fixation  the  system  will  continue  to  operate  within  the  50  degree  per 
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Figure  4.21 :  Scattergram  of  retinal  movement  for  subjects  RCL,  CSL,  SBR,  and 
ICR.  The  shaded  region  demarcates  the  limit  of  movement  for  an  arbitrarily 
chosen  retinal  landmark  during  a  45  second  retinal  movement  sequence  filmed 
using  subject  fixation.  The  diameter  of  the  shaded  region  did  not  exceed  1.8 
mm  for  subjects  RCL,  CSL,  and  SBR.  Subject  ICR’s  retinal  movement  was 
confined  to  two  similarly  sized  regions.  The  fixation  target  was  a  small  (.315 
cm  diameter),  red  light  emitting  diode.  No  rotational  or  scaling  movements 
were  observed  in  any  of  the  sequences. 
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Figure  4.22:  Time  record  of  the  40  x  40  pixel  search  area.  The  same  40  x  40 
pixel  area  was  “snapped'  from  a  video  sequence  of  subject  SBR's  retina. 

second  retinal  velocity  envelope.  If  this  retinal  velocity  is  exceeded  the  tracking 
algorithm  will  register  a  ‘lost  lock’  condition  and  then  attempt  to  re-establish 
lock. 

Visible  Tracking  Features  The  tracking  algorithm  described  in  this  section 
requires  visible  landmarks  (i.e.  vessels)  to  track  the  movement  of  the  retina. 
Chapter  7  describes  an  algorithm  to  track  the  movement  on  a  featureless  retina. 

Linearity  of  Retinal  Reflectance  The  templates  designed  for  the  track¬ 
ing  algorithm  depend  on  a  linear  relationship  between  the  fundus  illumination 
and  the  fundus  reflectance.  This  assumption  allows  tracking  to  continue  in  the 
prescence  of  an  unstable  illumination  source.  To  test  this  assumption  a  video- 
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fundus  camera  intensity  setting 

Figure  4.23:  The  linear  relationship  between  average  fundus  reflectance  and 
illumination 

tape  was  produced  of  human  retinal  movement  with  fixation  during  different 
levels  of  fundus  illumination.  The  average  image  gray  level  was  measured  over 
a  225  x  225  pixel  area  at  different  levels  of  illumination.  The  average  gray  level 
was  measured  over  a  large  imafje  area  to  minimize  the  effects  of  small  retinal 
movement  on  the  measurement.  Also,  the  average  gray  level  of  a  Labsphere 
.50  reflectance  standard  was  measured.  The  average  gray  level  was  measured 
in  a  25  x  25  pixel  area.  Ii:  both  cases,  a  linear  relationship  was  observed  once  a 
lower  illumination  threshold  was  exceeded.  Slight  deviations  from  the  linear  re¬ 
sponse  are  due  to  nonuniform  CCD  pixel  response  and  slight  retinal  movement 
from  frame-to-frame.  The  results  are  graphed  in  Figure  4.23. 


4.5.2  Geographic  Distributed  Normalized  Blood  Vessel  Templates 

The  first  step  in  developing  the  tracking  algorithm  was  designing  an  effective 
template  methodology.  The  template  developed  for  the  tracking  algorithm  is 
illustrated  in  Figure  4.24. 

This  template  has  the  following  features: 

•  The  template  requires  the  use  of  only  four  pixel  gray  levels. 

•  The  template  has  the  flexibility  of  variable  width.  This  idea  was  adapted 
from  Markow’s  template  [2]. 

•  The  template  provides  a  positive  response  at  only  a  single  coordinate. 
This  prevents  the  possibility  of  generating  a  false  positive.  Reference 
Figure  4.25. 

•  The  template  is  normalized.  Ideally,  it  should  provide  the  same  response 
under  varying  fundus  illumination  levels. 


The  template  response 

The  template  response  is  provided  by: 

template  response  =  (pi  -  p2  -  P3  +  p4)/(pl  +  p2  +  p3  +  p4)  (4.12) 

For  an  ideal  edge  (a  black  (0)  vessel  on  a  white  (255)  retinal  background)  the 
template  response  is: 


template  response  =  (255  -  0  -  0  +  255)/(255  +  0  +  0  +  255)  =  1  (4.13) 
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Figure  4.24:  The  tracking  template.  Six  of  the  one-dimensional  templates  are 
used  to  build  a  single  two-dimensional  template.  A  pair  of  horizontal  and 
vertical  one-dimensional  templates  are  selected  from  each  third  of  the  central 
portion  of  the  reference  image  demarcated  with  heavy  black  lines  above.  The 
horizontal  template  chosen  from  the  upper  third  of  the  reference  image  is  des¬ 
ignated  the  first  horizontal  template.  All  of  the  other  five  one-dimensional 
template  locations  are  referenced  to  the  first  horizontal  template’s  location. 
The  response  of  each  one-dimensional  template  is  normalized  as  shown  in  the 
normalized  template  response  equation. 
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Figure  4.25:  Normalized  template  response.  Response  of  the  normalized  tem¬ 
plate  in  the  vicinity  of  a  blood  vessel.  This  figure  illustrates  the  response  from 
a  single  template  in  the  vicinity  of  a  vertical  blood  vessel.  The  template  is 
swept  horizontally  across  the  vessel.  Different  horizontal  template  positions 
are  illustrated.  The  template  response  for  an  ideal  vessel  (a  black  (0)  vessel 
on  a  white  (255)  retina)  at  each  template  position  is  provided  on  the  template 
response  plot. 
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Figure  4.26:  Theoretical  template  response.  The  contrast  ratio  is  the  ratio  of 
vessel  gray  level  to  retinal  background  gray  level. 

Figure  4.26  provides  the  template  response  for  all  combinations  of  retinal  vessel 
and  background  gray  levels.  Note  that  the  peak  response  occurs  for  the  highest 
degree  of  dissimilarity  between  retinal  vessel  and  retinal  background  gray  levels. 


Automatic  selection  of  the  template  pairs. 

An  algorithm  was  developed  to  automatically  select  the  optimal  pairs  of  tem¬ 
plates  from  different  areas  of  the  reference  image.  The  overall  result  of  this 
template  selection  is  a  two  dimensional  template  which  has  an  optimal  tem¬ 
plate  pair  chosen  from  each  third  of  the  central  portion  of  the  reference  image. 
Reference  Figure  4.24. 

This  spatial  distribution  of  templates  reduces  false  positives.  If  template 
pair  selection  is  not  spread  across  the  reference  image,  all  of  the  template  pairs 
are  clustered  about  the  same  small  image  area.  This  clustering  of  templates 
leads  to  false  positives  since  nearby  pixels  have  similar  template  response. 
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Figure  4.27:  The  template  array. 


Individual  template  pairs  are  found  by  scanning  one  dimensional  hor¬ 
izontal  and  vertical  templates  within  the  partitioned  reference  image.  The 
horizontal  and  vertical  template  providing  the  greatest  numerical  response  is 
retained  as  the  optimal  pair.  This  scan  is  carried  out  in  each  third  of  the  ref¬ 
erence  image.  The  three  sets  of  optimal  template  pairs  are  then  locked  into 
a  single  two  dimensional  template  as  shown  in  Figure  4.24.  All  displacements 
are  referenced  to  the  horizontal  template  chosen  from  the  upper  one-third  of 
the  image.  This  template  is  designated  the  first  horizontal  template.  The 
complete  two-dimensional  template  is  stored  in  an  array  as  illustrated  in  Figure 
4.27. 

The  template  storage  function  allows  templates  for  individual  fields  of 
view  to  be  stored  in  the  same  patient  file  using  a  hashing  function  similar  to  that 
used  by  the  Lesion  Data  Base  storage  function.  Template  offset  values  from 
the  first  horizontal  template  along  with  individual  template  response  values  are 
stored. 

Implementation  of  a  limited  exhaustive  search. 

The  key  component  of  approaching  a  100  percent  success  rate  in  the  tracking 
algorithm  is  implementation  of  an  exhaustive  search.  Due  to  the  exorbinant 
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computation  cost  an  exhaustive  search  of  the  entire  image  for  each  new  frame 
is  not  feasible.  However,  a  limited  exhaustive  search  may  be  implemented  at 
a  significantly  lower  computation  cost.  The  key  idea  of  the  limited  search  is 
to  only  search  a  40  x  40  pixel  region  centered  about  the  last  known  correct 
match.  Based  on  previously  presented  calculations,  the  new  correct  match 
should  be  within  this  40  x  40  pixel  area.  The  new  match  coordinate  becomes 
the  center  of  the  40  x  40  pixel  search  area  in  the  subsequent  search.  The  x  and  y 
pixel  displacement  between  subsequent  match  coordinates  provides  the  amount 
of  retinal  movement.  This  information  is  interfaced  to  the  Laser  Positioning 
Subsystem  to  maintain  the  laser  on  the  required  lesion  coordinate. 

Since  six  separate  spatially  distributed  templates  (3  horizontal  and  3 
vertical)  locked  into  a  fixed  two-dimensional  orientation  are  used  to  track  reti¬ 
nal  movements,  a  separate  40  x  40  pixel  search  area  is  defined  about  each  of 
the  templates  as  diagrammed  in  Figure  4.28.  The  Matrox  PIP-1024  frame 
grabber  used  in  this  project  has  the  capability  to  efficiently  transfer  these  six 
image  windows  directly  into  the  host  computer.  Therefore,  the  computation  of 
match  location  is  performed  in  the  host  computer  which  has  significant  ‘num¬ 
ber  crunching’  capability.  This  window  transfer  mitigates  the  data  bottleneck 
constraint  identified  by  Markow  [2]. 

Although  the  calculation  of  the  match  coordinate  is  the  heart  of  the 
tracking  algorithm,  it  is  only  one  of  the  many  tasks  performed  by  the  algorithm. 

4.5.3  The  tracking  algorithm 

Early  in  this  chapter  a  flow  chart  of  the  tracking  algorithm  has  been  provided 
as  an  overview.  In  this  section  the  entire  algorithm  will  be  revisited  in  much 
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Tracking  template 

Retinal  vessels 


Fundus  camera  field  of  view  at  50* 

Figure  4.28:  The  limited  exhaustive  search 


Limited  exhaustive  search 
centered  about  last  known 
location  of  template 


greater  detail.  A  detailed  flow  chart  of  the  tracking  algorithm  is  provided  in 
Figure  4.29.  Reference  to  this  chart  during  the  following  description  may  prove 
helpful. 
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System  initialization 

The  RETINA  Tracking  Algorithm  begins  with  several  function  calls  to  initialize 
the  system.  Function  initialize.system  initializes  the  Matrox  PIP- 1024  video 
frame  grabber,  the  Data  Translation  DT2801A  data  acquisition  board,  and  the 
RETINA  Hardware/Software  (HW/SW)  Interface  to  a  known  state.  Also,  the 
following  status  flags  are  initialized:  lesion  .complete  to  one,  laser_point_error 
to  zero,  lost  Jock  to  zero,  patient.panic  to  zero,  last  Jesion  to  zero,  firstJesion 
to  one,  snapjieeded  to  one,  and  the  loop  control  variable  continue  to  one. 

Input  look  up  table  modification 

In  Chapter  3  a  method  of  modifying  the  frame  grabber’s  input  look  up  table 
to  enhance  image  contrast  was  discussed.  Recall  that  the  method  described 
does  not  incur  a  time  penalty.  After  system  initialization  the  user  is  prompted 
for  input  look  up  table  modification.  Thresholds  to  modify  the  look  up  table 
may  be  automatically  selected  by  the  algorithm  or  be  manually  selected  by  the 
user.  The  automatic  selection  examines  a  reference  image  histogram  for  the 
first  and  last  non-zero  entry  to  define  the  current  lower  and  upper  thresholds. 
The  new  lower  and  upper  thresholds  are  defined  as  0  and  255  respectively. 


RETINA  Tracking  Algorithm 
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Figure  4.29:  The  tracking  algorithm. 


100 


c:\sfb_0003  successfully  opened. 


1 

2 

3 

4 

5 

6 

7 

8 

9 

10 
11 
12 

13 

14 

15 

16 


LEFT  EYE  DATA: 


Diabetic  Detachment  Template 


0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 


0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 


0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 


RI7HT  EYE  DATA: 


Diabetic  Detachment  Template 


Choose  a  retinal  field  of  view  (fov)  for  treatment 
with  both  lesion  and  template  data  present. 

Select  an  eye  with  'r'  or  '1'  or  'x'  for  exit.  Press  i Enter ] . 

Figure  4.30:  Patient  data  availability.  AT"  entry  in  the  matrix  indicates  the 
presence  of  a  data  base  or  template.  In  this  example  patient  file  sfb_0003  has 
a  Lesion  Data  Base  stored  for  the  treatment  of  diabetic  retinopathy  in  field  of 
view  1  for  the  right  eye.  A  two-dimensional  tracking  template  is  also  stored  for 
this  field  of  view. 


Availability  of  the  patient  file 

After  system  initialization,  the  user  is  prompted  for  the  patient’s  file  designator. 
The  designator  consists  of  the  patient’s  three  initials  followed  by  an  underscore 
and  the  last  four  digits  of  the  patient’s  ..ocial  security  number.  For  example, 
my  patient  file  designator  would  be  sfb_8253.  The  patient  file  may  be  resident 
on  the  hard  drive,  a  floppy  disk,  or  even  a  magnetic  tape  cartridge. 

The  program  then  searches  through  the  specified  patient  file  and  pro¬ 
vides  a  matrix  of  the  available  field  of  view  data  for  treatment.  An  actual 
sample  of  this  matrix  is  provided  in  Figure  4.30.  AT’  indicates  the  presence 
of  specific  field  of  view  data  in  the  patient  file. 

The  user  is  then  prompted  for  the  eye  and  specific  field  of  view  for 


treatment.  If  the  field  of  view  the  user  wants  to  treat  does  not  have  both 
template  and  lesion  data  present,  the  user  is  given  the  opportunity  to  exit  the 
tracking  algorithm  or  select  another  field  of  view  for  treatment. 
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Once  a  field  of  view  is  selected  for  treatment,  the  RETINA  HW/SW  In¬ 
terface  automatically  illuminates  the  proper  fixation  light  emitting  diode  (LED) 
for  the  conjugate  eye.  A  detailed  description  of  this  interface  controller  is  pro¬ 
vided  later  in  this  document. 

After  the  appropriate  fixation  LED  is  illuminated,  the  required  template 
for  the  specified  field  of  view  is  loaded  in  the  global  variable  template-array. 
The  user  is  then  prompted  for  the  name  of  the  output  file  to  store  results. 
This  file  is  for  algorithm  development  purposes  only.  It  is  not  required  for  the 
clinical  system. 

Following  specification  of  the  output  file,  the  lesion  data  portion  of  the 
patient  file  is  accessed.  Recall  that  access  is  accomplished  using  a  hashing 
procedure.  The  user  is  then  prompted  to  turn  on  the  video  cassette  recorder 
In  the  clinical  system,  this  is  the  step  where  the  patient  would  be  instructed 
to  fixate  on  the  illuminated  LED  with  the  conjugate  eye.  As  soon  as  the  user 
depresses  [Enter]  the  actual  time  critical  tracking  algorithm  begins. 

The  time  critical  tracking  loop 

The  first  step  of  the  time  critical  tracking  loop  is  to  print  the  local  time  (to 
the  closest  millisecond)  to  the  outputfile.  This  «  developmental  requirement 
only.  It  allows  calculation  of  critical  tracking  loop  execution  time. 

The  lesion.complete  flag  is  then  checked.  Recall  that  this  flag  was  ini- 
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tially  set  to  one  so  upon  initial  loop  entry  the  lesion  .complete  steps  are  per¬ 
formed.  If  the  lesion.complete  flag  is  one,  the  patient  and  lesion  status  flags 
and  status  flip-flops  within  the  RETINA  HW/SW  Interface  are  reset.  The 
next  lesion  coordinate  is  then  obtained  from  the  patient  file.  If  this  is  the  first 
time  through  the  tracking  loop  the  first  lesion  coordinate  is  loaded.  The  x 
and  y  pixel  displacement  is  then  calculated  between  the  coordinate  of  the  first 
horizontal  template  and  the  lesion  coordinate.  Recall  that  all  coordinates  are 
referenced  to  the  first  horizontal  template.  Initial  lock  is  then  established  by 
exhaustively  searching  a  100  x  100  pixel  area  around  the  coordinate  of  the  first 
horizontal  template.  Lock  is  established  using  only  a  single  template  pair  in 
an  effort  to  conserve  computation  time  expenditure. 

After  initial  lock  is  established  the  lesion  data  just  retrieved  from  the 
data  file  is  compared  to  the  hash  markers  signaling  the  end  of  lesion  data.  If 
the  lesion  data  is  complete,  the  last  Jesion  status  flag  is  set  to  one  and  the  loop 
control  continue  variable  is  set  to  zero.  The  loop  is  then  exitted. 

If  the  lesion  data  is  not  complete,  the  lost-lock  status  flag  is  checked. 
If  it  is  set  one,  the  lost-lock  flag  is  reset  and  the  function  establish -lock  is 
called.  This  is  the  same  function  described  above  to  establish  initial  lock.  The 
function  is  provided  the  last  known  coordinate  of  the  first  horizontal  template. 
The  function  returns  the  coordinate  of  the  first  horizontal  template  after  lock 
is  established.  If  the  lost-lock  flag  is  not  set,  the  tracking  algorithm  continues. 

The  moving  video  sequence  is  then  ‘snapped’  if  the  snap-needed  flag  is 
set.  In  other  words  a  freeze  frame  is  obtained  to  calculate  the  current  template 
position.  The  function  calculate_templateJocation  calculates  the  current  posi¬ 
tion  of  the  first  horizontal  template  using  all  three  pairs  of  templates  in  a  40  x  40 
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pixel  exhaustive  search.  This  search  is  centered  about  the  last  known  location 
of  the  first  horizontal  template.  The  function  also  updates  the  global  variables 
for  the  correlation  value  obtained  for  each  of  the  six  individual  templates.  The 
results  are  then  printed  to  the  output  file. 

Loss  of  Lock 

After  the  position  of  the  first  horizontal  template  is  updated,  the  values  of  the 
six  individual  template  correlations  are  examined  for  a  loss  of  lock  condition. 
Each  of  the  individual  template  responses  are  normalized  by  the  anticipated 
template  value.  This  value  is  stored  in  the  patient  file  during  template  con¬ 
struction.  Any  loss  of  lock  conditions  may  be  programmed.  After  studying 
several  tracking  sequences  the  following  conditions  are  programmed  to  result 
in  loss  of  lock  status: 

•  If  any  four  of  the  six  templates  respond  with  less  than  15  percent  of  their 
anticipated  value,  or 

•  If  any  template  responds  with  greater  than  130  percent  of  its  anticipated 
value. 

If  a  loss  of  lock  condition  occurs  the  laser  shutter  is  closed  and  the  lost  Jock 
flag  is  set. 

Test  for  lesion  complete  and  patient  panic  status 

The  Reflectance  Based  Feedback  Control  System  will  monitor  lesion  growth  and 
issue  a  lesion  complete  signal  at  the  appropriate  time.  To  simulate  this  signal 
for  system  testing  a  pushbutton  switch  is  provided  on  the  RETINA  HW/SW 
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Interface.  When  depressed  the  lesion  complete  flip-flop  within  the  interface  is 
set.  The  software  function  checkJesion.status  checks  the  status  of  the  flip-flop. 
If  the  flip-flop  is  set,  the  software  flag  lesion  .complete  is  set.  This  condition 
causes  the  laser  shutter  to  close.  Patient  panic  is  tested  in  a  similar  manner. 

Laser  Position  Update 

If  the  algorithm  is  in  lock  and  the  lesion  is  not  complete  the  updated  laser 
position  is  calculated  and  the  laser  is  repositioned  via  the  Laser  Pointing  Sub¬ 
system.  The  laser  shutter  is  then  opened  (if  not  already  opened). 

Although  the  Laser  Pointing  Subsystem  will  have  closed  loop  feedback 
control  via  mechanisms  described  in  Chapter  6,  the  tracking  algorithm  also 
checks  if  the  laser  reached  its  intended  coordinate.  This  check  is  accomplished 
by  function  check  .laser-position  by  examining  a  12  x  12  pixel  region  centered 
around  the  prescribed  laser  coordinate  for  the  brightest  pixel.  The  centroid  of 
the  laser  spot  is  assumed  to  produce  the  brightest  pixel.  If  the  brightest  pixel 
is  found  at  the  intended  laser  final  destination  coordinate  no  laser  position 
update  is  required.  However,  if  the  brightest  pixel  is  found  within  the  12  x 
12  pixel  region  at  a  coordinate  other  than  the  intended  laser  final  destination 
coordinate,  a  correction  signal  is  provided  to  the  Laser  Pointing  Subsystem 
via  functions  moveJaserJiorizontally  and  moveJaser.vertically.  If  no  pixel  of 
laser  spot  brightness  is  found  within  the  12  x  12  pixel  region,  the  laser  shutter 
is  closed,  a  Laser  Pointing  Subsystem  malfunction  warning  is  issued,  and  the 
tracking  algorithm  is  aborted.  These  laser  position  correction  activities  are 
illustrated  in  Figure  4.31. 

A  Laser  Pointing  Subsystem  malfunction  could  be  caused  by  system 
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Figure  4.31:  Laser  position  check.  The  tracking  algorithm  checks  the  final 
destination  of  the  laser  via  function  check_laser_position.  This  function  exam¬ 
ines  a  12  x  12  pixel  region  centered  around  the  prescribed  laser  coordinate 
for  the  brightest  pixel.  If  the  brightest  pixel  is  found  at  the  intended  laser 
final  destination  coordinate  no  laser  position  update  is  required.  However,  if 
the  brightest  pixel  is  found  within  the  12  x  12  pixel  region  at  a  coordinate 
other  than  the  intended  laser  final  destination  coordinate,  a  correction  signal 
is  provided  to  the  Laser  Pointing  Subsystem  (left).  If  no  pixel  of  laser  spot 
brightness  is  found  within  the  12  x  12  pixel  region,  the  laser  shutter  is  closed, 
a  Laser  Pointing  Subsystem  malfunction  warning  is  issued,  and  the  tracking 
algorithm  is  aborted. 


misalignment  which  will  be  described  in  detail  in  Chapter  6,  therapeutic  laser 
failure,  or  failure  of  a  component  within  the  Laser  Pointing  Subsystem. 
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Continued  Processing 

The  time  critical  loop  is  then  continued  until  the  lesion  .complete  flag  is  set,  the 
patient  panics,  or  the  lesion  coordinates  for  the  field  of  view  under  treatment 
is  exhausted. 

Eye  Blinks 

When  a  patient  blinks,  the  tracking  algorithm  must  detect  the  condition  and 
close  the  laser  shutter.  The  system  must  return  to  tracking  after  the  blink 
episode  is  complete.  When  a  patient  blinks,  an  image  of  the  patient’s  eyelid  is 
provided  to  the  tracking  algorithm.  The  eyelid  is  featureless  and  results  in  loss 
of  lock.  The  loss  of  lock  computation  previously  described  handles  the  blink 
condition. 

Fail  Safe  Mechanism 

Fail  safe  mechanisms  are  provided  by  the  patient  panic  switch  and  the  internal 
protection  provided  with  the  laser  shutter.  In  the  event  of  a  power  failure  the 
laser  shutter  closes  and  will  not  reopen  until  manually  reset. 


Chapter  5 


The  Laser  Pointing  Subsystem 

5.1  Objective 

The  objective  of  the  Laser  Pointing  Subsystem  is  to  maintain  the  irradiating 
therapeutic  laser  on  the  prescribed  lesion  coordinate.  This  subsystem  must 
have  the  capability  to  redirect  the  laser  to  correct  for  retinal  movement  detected 
by  and  compensated  for  by  the  Retinal  Tracking  Subsystem. 

5.2  Ideal  Laser  Pointing  Subsystem  Characteristics 

To  accomplish  this  objective  the  Laser  Pointing  Subsystem  must  have  the  fol¬ 
lowing  capabilities: 

•  Be  able  to  redirect  the  laser  to  within  any  coordinate  within  the  visible 
fundus  camera  retinal  field  of  view.  The  fundus  camera  will  be  used  with 
the  50  degree  field  of  view  during  the  tracking  operation.  Therefore,  the 
Laser  Pointing  Subsystem  must  be  able  to  redirect  the  laser  to  within 
any  point  within  a  ±25  degree  field  of  view  Cartesian  coordinate  system 
as  illustrated  in  Figure  5.1, 

•  Have  response  characteristics  to  effectively  compensate  for  eye  move¬ 
ments, 
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0  x  axis  511 


Figure  5.1:  The  Fundus  Field  of  View  Cartesian  Coordinate  System 


•  Have  the  capability  to  safely  redirect  laser  energies  at  levels  used  to  form 
therapeutic  lesions, 


•  Have  a  feedback  mechanism  to  allow  for  closed  loop  control, 


•  Have  a  linear  response, 


•  Be  reliable  and  easy  to  maintain  and  align,  and 


•  Not  hinder  the  operation  of  other  system  components. 


5.3  Previous  Work 

Considerable  work  on  a  pointing  system  for  precise  laser  positioning  on  the 
human  retina  has  been  accomplished  by  Mainster,  Webb,  Timberlake,  Hughes, 
and  Pomerantzeff  in  the  development  of  the  Scanning  Laser  Ophthalmoscope. 
A  thorough  review  of  their  work  was  provided  in  Chapter  3.  In  their  appli- 
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cation  a  raster  scan  was  projected  on  the  retina  to  map  its  physical  features. 
Reflected  light  from  the  retina  was  captured  and  amplified  to  build  an  image 
of  the  retina.  The  laser  delivery  portion  of  their  design  may  be  modified  to 
serve  as  the  basis  for  the  Laser  Pointing  Subsystem.  The  key  components  of 
this  system  include  a  set  of  galvanometers  to  steer  the  laser  beam,  a  shutter 
for  laser  safety  control,  a  ’  opt)  to  integrate  the  Laser  Pointing  Subsystem 
with  the  Reflectance  Baseo  •eedback  Control  System.  No  attempt  is  made  to 
specify  a  final  design  for  the  Laser  Pointing  Subsystem.  Final  design  decisions 
must  be  based  on  the  actual  implementation  of  the  Reflectance  Based  Feed¬ 
back  Control  System  and  the  type  of  therapeutic  laser  system  chosen.  Work 
is  ongoing  to  study  the  feasibility  of  substituting  the  argon  laser  with  a  solid- 
state  laser  diode.  Instead,  equipment  considerations  driven  by  Retinal  Tracking 
Subsystem  requirements  are  presented.  Also,  a  developmental  Laser  Pointing 
Subsystem  is  provided.  This  system  was  used  to  test  the  closed  loop  tracking 
capability  of  the  Retinal  Tracking  Subsystem  and  used  for  in  vivo  demonstra¬ 
tions.  Information  will  be  provided  first  on  galvanometers  and  X-Y  scanning 
systems  followed  by  equipment  specifications.  Details  of  the  Laser  Pointing 
Subsystem  modifications  required  for  in  vivo  testing  are  deferred  to  Chapter  9. 

5.4  Galvanometers 

Galvanometers,  also  known  as  optical  scanners,  are  an  effective  method  of 
redirecting  a  laser  beam.  This  section  will  provide  a  brief  review  of  their  theory 
of  operation  followed  by  detailed  information  for  using  a  pair  of  galvanometers 
in  a  Cartesian  coordinate  scanning  system. 
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Figure  5.2:  The  galvanometer  [76] 


5.4.1  Theory  of  Operation 

Galvanometers  consist  of  an  iron  rotor  suspended  between  a  pair  of  front  and 
rear  bearings.  Two  permanent  magnets  and  two  drive  coil  windings  provide  for 
flux  between  the  rotor  and  four  stationary  poles.  The  magnitude  and  direction 
of  the  drive  coil  flux  determines  the  magnitude  and  direction  of  rotor  torque. 
Feedback  of  rotor  position  is  provided  by  a  pair  of  capacitors  formed  by  a  sta¬ 
tionary  slotted  metal  ring  attached  to  the  electrically  neutral  rotor.  The  slotted 
ring  forms  four  electrodes.  Each  pair  of  opposite  electrodes  forms  a  capacitor. 
The  capacitance  of  the  electrode  pairs  is  determined  by  rotor  position.  The 
current  (and  hence  rotor  position)  through  each  capacitor  is  monitored  when 
the  capacitor  is  excited  with  a  high  frequency  oscillator  [72,  73].  Reference 
Figure  5.2. 

Galvanometers  are  driven  by  external  driver  amplifiers.  Typically,  these 
driver  amplifiers  are  solid-state  variable  output  impedance  amplifiers.  These 
drivers  accept  a  DC  voltage  input  in  the  range  of  ±1.0  volt  and  provide  a 
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galvanometer  compatible  drive  current  proportional  to  the  input  voltage.  A 
stable  input  voltage  is  required  for  a  reliable  drive  current.  Also,  the  driver 
amplifier  must  be  supplied  with  a  stable  supply  voltage.  Typically  zero-offset 
and  gain  controls  are  provided  to  the  user  on  the  driver  amplifier  [74]. 

5.4.2  Characteristics 

Galvanometer  performance  and  characteristics  are  determined  by  a  number  of 
factors  including  armature  (rotor)  size,  inertia  of  the  mirror  being  driven,  and 
the  voltage  limits  of  the  external  driver  amplifier.  Galvanometers  are  provided 
in  an  open  loop  mode  with  no  position  feedback  or  in  a  closed  loop  variety  with 
position  feedback  using  the  capacitive  mechanism  described  above  [73].  Impor¬ 
tant  galvanometer  performance  parameters  include:  load-free  natural  resonant 
frequency,  peak-to-peak  mechanical  rotation,  armature  inertia,  torque,  linear¬ 
ity,  and  reliability. 

Frequency  Response 

Frequency  response  is  provided  as  a  load-free  natural  resonant  frequency  rat¬ 
ing.  Derating  curves  are  provided  to  determine  the  frequency  response  under  a 
given  load  condition.  Physically  smaller  galvanometers  have  a  higher  frequency 
response  due  to  the  lower  armature  inertia.  However,  these  smaller  galvanome¬ 
ters  have  lower  torque  capability.  Galvanometers  for  laser  beam  steering  are 
available  with  load-free  natural  resonant  frequencies  from  130  to  2,400  Hz  [75]. 

The  frequency  response  has  an  inverse  relationship  with  the  mirror  size 
for  a  given  galvanometer.  Generally,  the  mirror  inertia  should  be  between  .1  and 
10  times  the  armature  inertia.  At  larger  mirror  to  armature  inertia  ratios  the 


frequency  response  degrades  rapidly.  Both  mirror  inertia  and  armature  inertia 
are  typically  provided  in  gm  —  cm 2 .  Armature  inertia  ratings  are  available  in 
ranges  from  0.013  to  4.0  gm  —  cm 2  [75]. 
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Peak-to-Peak  Rotation 

Another  important  characteristic  of  the  galvanometer  is  peak-to-peak  mechan¬ 
ical  rotation.  This  is  typically  given  in  degrees.  Rotation  specifications  are 
alternatively  provided  in  optical  scan  angle.  Galvanometers  are  available  in 
optical  scan  angles  ranging  from  2  to  100  degrees  [75]. 

Linearity 

The  linearity  of  a  galvanometer  is  expressed  as  a  percentage  indicating  the 
ratio  of  maximum  position  signal  error  to  the  peak-to-peak  value  of  the  po¬ 
sition  signal.  Typical  values  range  from  ±0.3  degrees  to  ±1.0  degrees  [73]. 
Galvanometers  with  lower  linearity  ratings  (smaller  percentage)  are  more  ex¬ 
pensive. 

Failure  Modes  and  Reliability  Ratings 

Galvanometers  are  susceptible  to  failure  in  two  different  areas:  failure  of  elec¬ 
tronic  components  and  bearing  wear.  Reliability  ratings  are  provided  in  the 
number  of  failures  per  106  operating  hours.  Reliability  for  a  commercial  grade 
galvanometer  operating  at  25  degrees  centigrade  is  3.3  failures  per  106  hours. 
The  failure  rate  approximately  doubles  for  an  operating  temperature  of  125 
degrees  centigrade.  Military  specification  (mil  spec)  galvanometers  are  avail¬ 
able  with  failure  rates  an  order  of  magnitude  less  than  the  commercial  grade 
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FIRST  SURFACE  MASS  BALANCED 

Figure  5.3:  Scan  head  geometries.  Left  to  right:  first  surface  mirrors,  mass 
balanced  mirror  [76] 

galvanometer.  However,  their  cost  are  approximately  double  compared  to  a 
similar  commercial  grade  galvanometer  [72]. 

5.4.3  Scan  Heads 

Galvanometers  may  be  fitted  with  a  variety  of  scan  heads  including  mirrors  or 
beam  splitters.  There  are  a  number  of  variables  which  must  be  considered  in 
choosing  the  appropriate  scan  head  including:  optical  aperature,  optical  resolu¬ 
tion,  and  speed.  In  general,  larger  aperatures  provide  better  optical  properties 
but  lower  speed  capability  [76]. 

The  scan  head  aperature  is  the  opening  through  which  the  optical  sys¬ 
tem  will  image.  Optical  quality  is  specified  via  mirror  flatness  standards, 
scratch  and  dig  specifications,  and  reflectivity  parameters.  Different  mirror 
coatings  are  used  to  achieve  required  optical  quality  and  power  specifications. 
Mirrors  with  peak  irradiance  ratings  up  to  500  W/cm 2  are  available  [76]. 

There  are  two  different  types  of  scan  head  geometries:  first  surface 
mirrors  and  mass  balanced  mirrors.  Reference  Figure  5.3.  A  first  surface  mirror 
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has  the  reflective  surface  on  the  axis  of  rotation.  This  geometry  simplifies 
position  calculations.  However,  it  creates  a  lateral  imbalance  in  the  scanner 
when  the  mirror  rotates.  Mass  balanced  mirrors  have  mirror  mass  balanced 
about  the  axis  of  rotation.  This  type  of  geometry  allows  for  less  ‘robust'  mirrors. 
However,  this  configuration  is  more  sensitive  to  alignment  and  lineanty  errors 

[76]. 


5.5  X-Y  Scanning  Systems 

Manufacturer  configured  X-Y  scanning  systems  are  available.  These  systems 
use  2  mirrors  and  2  galvanometers  separately  driven  to  provide  an  optical  scan 
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in  an  X-Y  plane.  The  mirrors  are  placed  orthogonally  with  the  smaller  mirror 
mounted  lower  to  provide  the  X  scan  and  the  larger  mirror  mounted  above  for 
the  Y  scan.  The  laser  beam  follows  the  path  to  the  target  as  illustrated  in 
Figure  5.4  [76]. 

5.5.1  Drive  Signals 

Drive  signals  are  separately  provided  to  the  X  and  Y  scanner  drivers  to  position 
the  laser  at  any  point  in  the  X-Y  plane.  These  voltage  drive  signals  may  be  of 
three  different  types:  a  raster  signal,  a  step  signal,  or  a  vector  signal  [76]. 

A  raster  signal  is  used  to  scan  the  laser  beam  in  a  raster  fashion  in  the 
X-Y  plane.  To  generate  a  raster  scan  the  X  driver  amplifier  is  provided  a  high 
frequency  sawtooth  wave  to  scan  the  raster  across  the  X-Y  plane.  The  Y  driver 
amplifier  is  provided  a  slower  signal  to  increment  the  laser  scan  a  line  at  a  time. 
This  type  of  scan  pattern  was  used  with  the  Scanning  Laser  Ophthalmoscope 
[76], 

The  step  signal  allows  positioning  of  the  laser  beam  to  any  random  po¬ 
sition  within  the  X-Y  plane.  This  method  of  laser  positioning  has  an  inherently 
long  settling  time.  This  is  the  time  required  to  settle  within  a  small  final  area 
and  compensate  for  any  small  position  errors  that  might  exist  after  99  percent 
of  the  scanner  motion  is  complete.  Settling  time  increases  with  scan  head  di¬ 
ameters.  A  small  5  mm  scan  head  has  a  settling  time  of  6.5  ms  with  a  1.0  ms 
travel  time.  A  large  50  mm  scan  head  has  a  58.5  ms  settling  time  with  a  40.0 
ms  travel  time  [76]. 

Vector  rotation  is  generated  by  the  coordinated  movement  of  both  the 
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X  and  Y  galvanometer.  Vector  rotation  may  use  the  point-to-point  or  the 
skywriting  technique.  In  the  point-to-point  method  vectors  begin  and  end  at 
drawn  line  endpoints.  This  is  the  fastest  method  of  point-to-point  transit.  The 
skywriting  method  backs  the  scanner  position  up,  scans  across  the  vector  start 
point,  past  the  vector  end  point,  and  then  stops  beyond  the  end  point.  This 
method  is  slower  than  point-to-point  vectoring  but  solves  scanner  acceleration 
and  deceleration  problems  [76]. 

5.5.2  Sources  of  Error 

Ideally,  the  laser  should  be  precisely  directed  to  any  point  in  the  X-Y  plane. 
However,  due  to  various  sources  of  error  this  is  not  always  the  case.  Sources  of 
error  include  galvanometer  errors  and  image  distortion  errors. 

Galvanometer  Errors 

Galvanometer  related  errors  include  wobble,  zero-drift,  and  repeatability  errors. 
Wobble  errors  are  rotations  orthogonal  to  the  axis  of  desired  rotation.  Wobble 
is  caused  primarily  by  mirror  imbalance.  It  is  caused  to  a  lesser  extent  by 
asymmetrical  forces  on  the  galvanometer  rotor  or  from  worn  bearings  [73]. 

Zero-drift  error  is  a  slow  variation  of  the  X  axis  rest  position  with  time. 
This  error  results  from  drive  signal  variation  with  temperature.  This  type  of  er¬ 
ror  may  be  minimized  by  applying  temperature  regulation  to  the  galvanometer 
housing  [73]. 

Repeatability  errors  is  a  variation  of  the  X  axis  rest  position  from  one 
scan  to  the  next.  Several  noise  sources  contribute  to  the  repeatability  error. 
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The  largest  error  source  of  this  type  is  bearing  noise.  Bearing  noise  occurs  as 
a  random  fluctuation  in  bearing  performance.  Performance  is  affected  by  rotor 
speed,  bearing  condition,  bearing  lubricant  condition,  and  dust  contamination 

[73]. 

Image  Distortion  Errors 

Image  distortion  errors  are  of  two  types:  optical  path  distortions  (OPD)  and 
cosine  distortion.  Optical  path  distortions  are  caused  by  mirror  quality  imper¬ 
fections.  This  type  of  error  is  usually  caused  by  mirror  non-flatness.  It  results 
in  the  laser  beam  coming  to  a  focus  in  an  elongated  or  distorted  spot  [76]. 

Cosine  distortion  is  caused  by  the  geometry  of  the  X-Y  scan  system.  It 
is  due  primarily  to  the  scanner  pointing  across  a  wide  angular  range  from  a 
single  point.  This  results  in  an  elongation  of  the  incident  laser  beam  at  the  far 
reaches  of  the  scan  field.  Cosine  distortion  also  results  in  an  elongated  laser 
spot.  Cosine  distortions  are  small  for  a  laser  pointing  system  due  to  the  small 
laser  spot  size.  Cosine  distortion  may  be  minimized  by  avoiding  the  far  reaches 
of  the  image  plane  [76]. 

5.6  Retinal  Tracking  Subsystem  Requirements 

Only  three  Laser  Positioning  Subsystem  parameters  are  driven  by  the  Retinal 
Tracking  Subsystem.  These  are:  response  time,  position  resolution,  and  the 
scan  type  employed. 
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5.6.1  Response  Time 

Retinal  Tracking  Subsystem  real  time  specifications  are  provided  in  Chapter 
10.  These  specifications  indicate  to  track  with  200  micron  target  radius  at  a 
retinal  velocity  of  50  degrees/ second  requires  a  position  update  approximately 
120  times  per  second.  This  requires  a  galvanometer  with  a  loaded  frequency 
response  of  120  Hz.  Galvanometers  with  this  capability  are  readily  available. 

5.6.2  Position  Resolution 

The  resolution  of  the  Retinal  Tracking  Subsystem  is  determined  by  the  resolu¬ 
tion  of  the  CCD  video  camera  coupled  to  the  fundus  camera.  The  developmen¬ 
tal  system  configuration  has  an  approximate  retinal  surface  resolution  of  40 
microns.  The  is  equivalent  to  .137  degrees.  Galvanometers  with  this  capability 
are  readily  available. 

5.6.3  Maximum  Displacement 

The  Retinal  Tracking  Subsystem  employs  a  fundus  camera  with  a  50  degree 
field  of  view.  This  requires  a  pair  of  X-Y  galvanometers  with  a  50  degree  peak- 
to-peak  optical  scan  angle  capability.  Galvanometers  with  this  capability  are 
readily  available.  Galvanometers  with  an  eight  degree  peak-to-peak  optical  scan 
angle  were  employed  in  the  development  system.  Increasing  the  displacement 
from  laser  mirror  to  target  countered  the  limited  deflection. 
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5.6.4  Closed  Loop  Control 

As  previously  mentioned,  some  galvanometers  are  equipped  with  position  feed¬ 
back  capability  using  rotor  position  capacitive  monitors.  Specially  config¬ 
ured  driver  amplifiers  are  equipped  to  process  the  feedback  information  from 
the  galvanometer  to  ensure  precise  galvanometer  positioning.  Galvanometers 
and  driver  amplifiers  of  the  closed  loop  type  may  be  used  with  the  Laser 
Positioning  Subsystem  to  provide  an  additional  level  of  safety  beyond  the 
checkJaser_position  function  described  in  Chapter  4. 

5.6.5  Scan  Type  Employed 

To  provide  for  timely  movement  of  the  therapeutic  laser  the  vector  point-to- 
point  scan  type  will  be  used  with  the  Laser  Positioning  Subsystem. 

5.7  System  Design 

A  X-Y  scan  system  with  the  above  specifications  is  readily  available  from  Gen¬ 
eral  Scanning  Incorporated  for  approximately  four  thousand  dollars.  The  bud¬ 
get  for  the  development  system  study  did  not  allow  purchase  of  this  X-Y  scan 
system.  Instead,  galvanometers  and  driver  amplifiers  already  available  within 
the  laboratory  were  used  to  demonstrate  the  system  concept. 

5.8  Development  System  Implementation 

A  detailed  description  of  the  developmental  system  instrumentation  is  pro¬ 
vided  in  Chapter  7  of  this  document.  This  system  uses  General  Scanning 
Incorporated  open  loop  galvanometers  and  driver  amplifiers  to  demonstrate 
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Figure  5.5:  The  developmental  Laser  Positioning  Subsystem 


the  capabilities  of  the  Retinal  Tracking  Subsystem.  A  HeNe  laser  is  used  as 
the  simulated  therapeutic  laser.  This  laser  is  projected  over  two  meters  to  a 
simulated  retina  to  compensate  for  the  limited  8  degree  peak-to-peak  deflection 
of  the  galvanometers.  Reference  Figure  5.5. 

The  simulated  retina  is  simply  a  white  cardboard  screen.  An  image 
of  an  actual  human  retina  taken  through  a  fundus  camera  set  at  a  50  degree 
field  of  view  is  projected  onto  the  screen.  The  screen  is  imaged  with  the  CCD 
video  camera  of  the  Retinal  Tracking  Subsystem.  Simulated  retinal  movement 
is  accomplished  by  moving  the  retinal  image  in  reference  to  the  stationary 
screen.  Further  details  of  the  testing  apparatus  is  provided  in  Chapter  8. 
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5.8.1  System  Alignment 

Precise  alignment  is  required  between  the  Retinal  Tracking  Subsystem  and 
the  Laser  Positioning  Subsystem.  This  is  accomplished  by  using  a  common 
coaligned  coordinate  system.  The  coordinate  system  of  the  frame  grabber  is 
a  readily  available  common  format.  It  has  an  X  and  Y  axis  each  divided 
into  512  increments.  To  coalign  the  laser  to  the  same  coordinate  system,  the 
peak-to-peak  drive  voltage  required  by  the  driver-amplifier  is  divided  into  512 
separate  increments.  The  two  systems  are  then  aligned  by  generating  a  225 
x  225  pixel  test  pattern  with  the  Laser  Pointing  Subsystem  and  overlaying  it 
with  a  225  x  225  pixel  test  pattern  within  the  image  plane  of  the  frame  grabber. 
When  the  tracking  algorithm  computes  an  updated  laser  position  coordinate, 
the  updated  coordinate  from  the  tracking  algorithm  is  simply  passed  to  the 
functions  moveJaser_horizontally  and  moveJaser_verticalIy.  These  functions 
provide  the  proper  voltage  to  the  driver-amplifiers  via  the  data  acquisition  and 
control  board  to  move  the  laser  to  the  updated  coordinate.  It  should  be  noted 
that  this  is  easily  modified  for  other  configurations  by  modifying  the  software. 

The  drive  signal  used  in  the  development  system  is  a  vector  point-to- 
point  scan.  Correction  signals  for  the  X  and  Y  galvanometers  are  provided 
virtually  simultaneously  by  the  data  acquisition  and  control  system.  Although 
specifications  for  the  real  time  system  will  include  galvanometers  equipped  with 
position  feedback  mechanisms,  an  additional  feedback  mechanism  provided  by 
the  tracking  algorithm  for  redundancy  is  highly  recommended.  This  mechanism 
which  was  described  in  Chapter  4  will  be  tested  in  Chapter  8. 
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5.9  Laser  Pointing  Subsystem  Testing 

Results  of  testing  the  Laser  Pointing  Subsystem  are  provided  in  Chapter  8 
“Development  System  Testing” . 


Chapter  6 


Tracking  on  a  Featureless  Retina 

6.1  Alternate  tracking  mechanism  requirement 

Chapter  4  of  this  document  provided  a  method  to  track  and  compensate  for  the 
movement  of  the  retina  using  visible  retinal  landmarks  as  templates  to  derive 
position  information.  This  method  is  effective  in  tracking  retinal  movement 
as  long  as  visible  retinal  features  are  available.  In  retinal  fields  of  view  away 
from  the  optic  disk  the  retinal  vessels  become  less  populous.  Also,  both  hor¬ 
izontally  and  vertically  oriented  vessels  necessary  for  effective  vessel  template 
construction  may  not  be  present  in  these  peripheral  fields  of  view. 

This  situation  motivated  research  toward  an  alternate  tracking  method 
for  peripheral  fields  of  view  using  laser  lesions  as  landmarks  for  deriving  po¬ 
sitional  updates.  It  was  also  hoped  that  this  alternate  method  might  prove 
computationally  less  expensive  and  less  susceptible  to  variations  in  the  retinal 
illumination  source  than  the  vessel  tracking  algorithm. 

6.2  Overview 

This  chapter  describes  an  alternate  tracking  concept  using  therapeutic  laser 
lesions  as  templates.  The  chapter  begins  with  a  description  of  a  single  lesion 
template  and  the  two-dimensional  lesion  template.  Two  different  methods  of 


123 


124 


implementing  a  lesion  tracking  algorithm  are  then  discussed  with  their  inher¬ 
ent  advantages  and  disadvantages.  These  two  methods  are  called:  the  Unique 
Template  (UT)  method  and  the  Adaptive  Template  (AT)  method.  The  chap¬ 
ter  concludes  with  a  description  of  the  Lesion  Tracking  and  Image  Analyzer 
(LETINA)  software  developed  to  implement  and  test  the  lesion  tracking  meth¬ 
ods. 


6.3  The  Lesion  Template 

The  lesion  template  is  very  similar  to  the  vessel  template  described  in  Chapter 
4.  The  individual  horizontal  and  vertical  templates  are  normally  oriented  to 
one  another  and  referenced  to  the  center  reference  pixel.  This  template  configu¬ 
ration  forms  a  ‘crosshair’  over  a  lesion  site.  Reference  Figure  6.1.  Computation 
of  the  template  response  has  been  slightly  modified  to  account  for  searching 
for  a  lighter  laser  lesion  against  a  darker  retinal  background.  The  template 
response  is  given  by: 


template  response 


(pi  -f  p2  +  p3  +  p4)  -  (p5  +  p6  +  pi  +  p8) 
(pi  +  p2  +  p3  +  p4  +  p5  -I-  p6  +  p7  -f  p8) 


(6.1) 


This  template  provides  a  response  of  1  to  an  ideal  lesion.  An  ideal  lesion 
is  a  white  (255)  lesion  on  a  black  background(O).  Function  find  Jesion.templates 
was  devised  to  search  a  user  specified  image  search  area  for  all  occurrences  of 
lesion  templates.  This  function  exhaustively  searches  all  pixels  in  the  specified 
search  area  for  a  lesion  by  testing  templates  of  radius  1  pixel  through  5  pixels 
for  a  fit.  These  radii  correspond  to  lesion  diameters  of  150  to  500  microns 
respectively.  A  50  x  50  pixel  search  area  requires  approximately  90  seconds  of 


e  6.1:  The  lesion  template.  For  a  given  reference  pixel, 
1  to  5  pixels  are  tested  for  a  lesion  fit.  The  better  the  fi 
late  response. 
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exhaustive  search  area 

image  area  captured 

Figure  6.2:  The  lesion  template  search.  The  white  region  is  the  actual  search 
area.  The  gray  shaded  region  is  provided  as  a  pad  to  allow  for  the  search  of 
the  entire  white  region. 

processing  (486-33)  to  locate  all  potential  lesion  locations.  The  function  loads 
potential  lesion  templates  into  a  linked  list  ordered  from  largest  to  smallest 
template  response.  The  linked  list  insertion  routine  checks  to  ensure  that  a 
duplicate  template  is  not  inserted  in  the  list  and  that  only  the  lesions  with  the 
highest  responses  are  retained  for  two-dimensional  lesion  template  building. 
The  lesion  search  is  illustrated  in  Figure  6.2. 


6.4  Two-dimensional  lesion  templates 

Once  potential  lesion  templates  have  been  found,  the  user  is  prompted  for  the 
number  of  lesion  templates  desired  for  the  two-dimensional  lesion  template. 
The  function  auto.buildJesion.template  then  constructs  the  two-dimensional 
lesion  template.  All  individual  lesion  templates  are  referenced  to  the  individual 
lesion  template  with  the  highest  template  response  as  illustrated  in  Figure 
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Figure  6.3:  The  two-dimensional  lesion  template  formed  by  three  separate  le¬ 
sion  templates.  All  separate  templates  are  referenced  to  the  individual  template 
with  the  highest  template  response.  The  lesion  template  on  the  left  has  the 
highest  template  response  in  this  illustrative  example. 

6.3.  The  function  superimposes  a  black  cursor  on  the  lesions  used  in  the  two- 
dimensional  template.  The  two-dimensional  template  parameters  are  stored  in 
the  template  array  as  illustrated  in  Figure  6.4. 


6.5  Testing  on  ideal  lesions 

The  template  finding  and  automatic  building  functions  were  tested  on  a  series 
of  ideal  lesions.  A  test  pattern  of  three  lesions  was  constructed  using  function 
generate.test .pattern.  This  function  can  generate  any  arbitrary  user  specified 
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- template_array - 

Template  Constants  Lesion  Constants 

[0]  [1]  [2]  [3]  [4]  [5]  [6]  [7] 

x_off  y_off  x_dm  y_dm  x_rad  y_rad  corr  unused 
[0]  0  0  11  14  5  7  100  0 

[1]  -10  -10  9  12  4  5  100  0 

[3]  -20  -20  7  9  3  4  100  j 

Total  template  correlation:  300 

Figure  6.4:  The  template  array 


lesion  pattern.  The  finding  and  building  functions  correctly  located  and  built 
a  two-dimensional  template.  The  results  of  this  test  are  illustrated  in  Figure 
6.5 


6.6  Testing  lesion  templates  on  a  rabbit  retina 

Dr.  Maya  Jerath  performed  in  vivo  tests  on  cross  bred  Californian  and  New 
Zealand  rabbits  in  support  of  her  research  in  real  time  control  of  laser  induced 
retinal  lesions.  In  these  tests  3  kg  rabbits  were  anesthetized  intramuscularly 
with  a  combination  of  Ketamine  (35  mg/kg)  and  Rompan  (5.9  mg/kg).  The 
rabbit’s  eye  was  then  dilated  with  Atropine  and  a  speculum  inserted  to  open 
the  eyelid.  A  suture  was  placed  in  the  medial  rectus  muscle  to  allow  eye 
movement.  Controlled  lesions  were  then  placed  on  the  rabbit’s  retina  [4].  These 
experiments  were  filmed  using  a  CCD  video  camera  coupled  to  a  fundus  camera 
set  a  50  degree  field  of  view.  No  optical  enhancement  filters  were  used  in  the 
filming.  These  video  tapes  were  used  to  test  the  efficacy  of  the  lesion  template 
concept.  Several  excerpts  were  selected  from  the  tapes.  Selection  was  based  on 
the  presence  of  multiple  lesions  within  the  central  portion  of  the  field  of  view. 


Figure  6.5:  The  function  generate.test -pattern  was  used  to  generate  three  ideal 
(255)  lesions  of  different  diameters  on  a  black  (0)  background.  The  functions 
find  Jesion -templates  and  auto_buildJesion_templates  correctly  located  the  in¬ 
dividual  templates  and  constructed  a  two-dimensional  template.  The  templates 
used  in  the  two-dimensional  template  are  highlighted  with  a  black  cursor. 
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Two  excerpts  were  chosen  from  the  video  history  of  the  experiment 
performed  on  rabbit  RC.  One  excerpt  had  slow  retinal  movements  of  the  anes¬ 
thetized  rabbit  while  the  other  excerpt  had  rapid  suture  induced  retinal  move¬ 
ment.  Figure  6.6  illustrates  the  result  of  the  lesion  tracking  tests.  The  figure 
on  the  top  left  is  the  original  reference  image  used  to  construct  the  lesion  tem¬ 
plate  and  the  Lesion  Data  Base.  The  bright  elongated  object  in  the  upper  left 
portion  of  the  image  is  the  rabbit’s  optic  disk.  Four  lesions  are  present  in  the 
center  of  the  image.  Function  find  Jesion.templates  correctly  found  the  three  le¬ 
sions  with  the  highest  template  response.  The  highest  actual  template  response 
was  0.05.  Recall  that  an  ideal  lesion  template  has  a  response  of  1.  Function 
auto_build  Jesion -template  correctly  assembled  these  three  lesion  templates  into 
a  single  two-dimensional  template.  The  three  lesions  chosen  to  build  the  tem¬ 
plate  are  highlighted  with  a  black  cursor  in  the  top  right  image.  Function 
buildJesion_data_base  then  plotted  the  coordinates  of  the  desired  therapeutic 
lesions.  Results  of  tracking  normal  retinal  movement  are  shown  in  the  lower  left 
image.  Results  of  tracking  the  faster  suture  induced  movement  are  shown  in 
the  bottom  right  image.  The  tracking  algorithm  was  able  to  successfully  track 
these  faster  movements  for  a  15  second  span.  When  the  eye  was  quickly  pulled, 
the  upper  velocity  limit  of  the  tracking  algorithm  was  exceeded.  This  occurred 
at  an  approximate  retinal  velocity  of  20  degrees  per  second.  Precise  speed 
parameters  of  the  tracking  algorithm  will  be  provided  in  Chapter  8.  These 
experiments  demonstrated  the  feasibility  of  using  laser  lesions  as  landmarks  for 
tracking  retinal  movement. 


Figure  6.6:  Results  of  the  lesion  tracking  experiments  conducted  on  anes¬ 
thetized  pigmented  rabbits.  Top  left:  reference  image  used  to  construct  the 
lesion  templates.  Top  right:  results  of  building  a  two-dimensional  lesion  tem¬ 
plate  and  Lesion  Data  Building.  Templates  used  in  the  two-dimensional  tem¬ 
plate  are  highlighted  with  black  cursors.  Bottom  left:  results  of  tracking  the 
movement  of  the  anesthetized  subject  RC.  Bottom  right:  results  of  tracking 
movement  induced  with  a  suture  attached  to  the  medial  rectus  muscle. 
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6.7  Template  tracking  methods 

Two  different  methods  have  been  devised  to  employ  lesion  templates  as  tracking 
landmarks.  This  section  describes  each  method  in  turn  with  its  associated 
advantages  and  disadvantages. 

6.7.1  The  Unique  Template  tracking  method 

The  Unique  Template  tracking  method  uses  a  cluster  of  unique  lesions  to  form 
a  distinctive  template.  For  the  study  described  in  this  section,  a  triad  of  200 
micron  lesions  forming  an  isosceles  triangle  was  used.  Reference  Figure  6.7. 
Any  distinct  lesion  pattern  may  be  used.  The  reference  coordinate  for  the 
template  is  at  the  triangle  center.  Recall,  from  a  previous  chapter  that  the 
success  of  panretinal  photocoagulation  therapy  is  roughly  proportional  to  the 
retinal  area  covered  with  a  lesion.  The  triad  covers  approximately  one-half  the 
area  of  a  500  micron  lesion. 

The  lesion  triad  template  can  be  used  to  provide  ‘interlocking’  templates 
between  adjacent  fields  of  view.  For  example,  a  series  of  lesion  triad  templates 
could  be  placed  in  a  ring  outside  the  normal  therapeutic  lesions  as  illustrated 
in  Figure  6.8.  These  lesions  could  then  be  used  by  more  peripheral  fields  of 
view  as  tracking  templates  while  providing  therapeutic  value.  This  method  of 
interlocking  templates  may  be  extended  across  the  surface  of  the  retina.  This 
method  has  the  advantage  of  simplicity.  However,  this  simplicity  is  interrupted 
should  a  blood  vessel  cross  an  intended  position  of  a  lesion  triad  template.  The 
function  buildJesion_data_base  required  slight  modifications  to  plot  the  outer 
ring  of  interlocking  lesion  triad  templates.  The  result  of  these  modifications 
are  provided  in  Figure  6.9. 
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200  micron! 


500  microns 


Figure  6.7:  Left:  the  lesion  triad  template.  Right:  the  lesion  triad  template 
illustrated  to  scale  with  a  500  micron  lesion. 


Figure  6.9:  Results  of  adding  interlocking  lesion  triad  templates  to  the  function 
build  Jesion -data-base 

6.7.2  The  Adaptive  Template  tracking  method 

The  Adaptive  Template  tracking  method  provides  for  the  orderly  placement 
of  therapeutic  lesions  using  predecessor  lesions  as  a  template  to  form  current 
lesions.  This  methods  starts  with  a  lesion  in  the  center  of  the  field  of  view  for 
treatment.  This  first  lesion  serves  as  a  template  to  form  the  second  therapeutic 
lesion.  The  two  therapeutic  lesions  are  then  used  as  a  template  to  form  the 
third  lesion.  Once  the  third  therapeutic  lesion  is  formed  a  triad  template  is 
used  consisting  of  the  first  three  lesions  to  form  the  fourth  lesion.  This  process 
continues  in  a  repeatable  pattern  radially  outward  from  the  center  of  the  field 
of  view.  This  therapeutic  lesion  formation  process  is  illustrated  in  Figure  6.10. 
The  completed  pattern  containing  61  therapeutic  lesions  is  illustrated  in  Figure 
6.11.  A  pattern  of  61  500  micron  lesions  fills  the  central  region  of  a  50  degree 


retinal  field  of  view.  This  tracking  method  is  called  the  Adaptive  Template 
method  since  the  tracking  triad  template  is  updated  for  every  new  lesion. 
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This  tracking  method  has  inherent  advantages  and  disadvantages.  The 
triad  template’s  component  lesion  template  are  local  to  one  another  is  one 
advantage.  Therefore,  any  variation  in  the  retinal  illumination  source  should 
effect  the  component  lesion  templates  more  evenly  as  compared  to  vessel  tem¬ 
plates.  This  advantage  also  applies  to  the  UT  method.  The  AT  method  also  has 
the  additional  advantage  of  using  therapeutic  sized  lesions  as  lesion  templates. 
Recall  that  the  UT  method  required  lesions  distinguishable  from  therapeutic 
sized  lesions.  This  reduced  the  therapeutic  value  of  the  UT  template  lesions. 

One  disadvantage  of  the  Adaptive  Template  tracking  method  is  control¬ 
ling  the  formation  of  the  first  lesion.  All  subsequent  lesions  use  previous  lesions 
as  landmarks  to  stabilize  the  therapeutic  laser.  However,  the  first  lesion  must 
rely  on  some  other  method.  This  may  be  remedied  by  using  some  of  the  other 
tracking  methods  already  presented  in  this  document.  Another  disadvantage 
of  the  Adaptive  Template  tracking  method  is  that  each  lesion  template  triad 
formed  must  provide  a  separate  and  distinct  template  from  all  other  template 
triads  in  the  array.  Several  methods  rimy  be  used  to  provide  lesion  ‘distinct¬ 
ness’  in  the  array.  Adjusting  lesion  diameter  or  depth  would  provide  lesion 
variability;  however,  this  method  would  defeat  the  purpose  of  the  Reflectance 
Based  Control  System.  Another  method  of  providing  variability  is  to  provide 
a  slight  coordinate  shift  in  lesion  placement.  A  lesion  could  be  shifted  a  pixel 
or  two  in  a  given  direction  to  provide  distinct  lesions.  This  would  provide  the 
unique  lesion  template  triads  and  provide  only  a  slight  modification  in  the  le¬ 
sion  array.  To  study  the  impact  of  this  disadvantage  in  detail,  a  simulation 


137 


lesion  3 

/ 

/ 

lesion  1-"' 

lesion  2  lesion 

template 

lesion  template 

lesion  4~n 

A" 

lesion  template 

A7> 

mm 

- - j - ^ 

lesion  template 

lesion  5 

1 

lesion  <A" 

Hi 

V/ 

g&mrl 

"V 

Mmflra 1 

VcMml 

lesion 

lesion  7  temPlate 

^  lesion  8 

lesion  template 

Figure  6.10:  Therapeutic  lesion  formation  using  the  Adaptive  Template  method 
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Figure  6.11:  The  complete  pattern  of  61  therapeutic  lesions.  The  lesions  are 
shown  here  in  an  orderly  array  of  uniform  size.  To  track  retinal  movement  using 
lesions  in  a  triad  pattern,  some  method  of  ‘distinctness’  must  be  injected  into 
the  array  pattern.  Varying  the  diameter  or  depth  of  the  lesion  would  provide  the 
necessary  distinctness  but  would  contradict  the  goal  of  the  Reflectance  Based 
Feedback  Control  System.  Instead,  distinctness  is  introduced  by  randomly 
choosing  one  of  twelve  lesion  offsets.  These  slight  offsets  provide  the  necessary 
distinctness  without  significantly  altering  the  order  of  the  array. 


program  was  written  to  model  lesion  placement  and  triad  template  formation 
using  the  Adaptive  Template  method.  The  next  several  sections  describe  this 
study  in  detail. 
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Feasibility  simulation  testing 

To  model  the  efficacy  of  the  Adaptive  Template  method,  a  simulation  of  the 
method  was  written.  The  first  requirement  for  simulation  development  was  to 
determine  the  number  of  distinct  lesion  types  to  form  a  therapeutic  array  of  61 
lesions  without  forming  any  duplicate  lesion  triads.  Second,  it  was  necessary 
to  show  that  an  algorithm  could  be  developed  to  form  therapeutic  lesions  in 
an  orderly  method  using  predecessor  lesions  as  templates  to  form  the  current 
lesion. 

Determination  of  the  distinct  number  of  lesion  types  to  form  a  ther¬ 
apeutic  array  To  determine  the  number  of  distinct  lesion  types  required  to 
form  an  array  of  61  lesions  with  no  duplicate  lesion  triads,  a  program  called 
RANDOM  was  written.  RANDOM  loads  interconnect  data  on  all  lesion  triads 
within  a  61  lesion  array.  There  are  a  total  of  96  distinct  lesion  triads  formed  by 
the  61  lesions.  The  number  of  distinct  lesion  types  (n)  for  simulation  is  then 
provided  by  the  user.  A  random  number  generator  then  assigns  each  of  the  61 
lesions  with  a  distinct  type  from  1  to  n.  The  entire  complement  of  96  lesion 
triads  are  then  tested  for  duplicates.  Duplicate  information  is  then  provided. 

The  random  number  generator  A  random  number  generator  was 
used  to  assign  lesion  types  within  the  61  lesion  array  to  avoid  duplicate  triad 


entries.  A  random  number  generator  of  the  linear  congruential  generator  type 
was  used.  This  type  of  generator  provides  random  numbers  using  the  recurrence 
relation: 
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Ij+ 1  =  alj  -j-  c  ( mod  m)  (6.2) 

where  a  is  the  multiplier,  c  the  increment,  and  m  the  modulus.  The 
initial  value  of  Ij  is  called  the  seed.  The  seed  establishes  the  starting  point  of 
the  repeating  sequence.  To  break  up  the  sequential  correlation  of  the  random 
stream,  a  shuffling  routine  is  used.  The  shuffling  routine  uses  the  current 
random  number  to  select  a  random  number  from  an  array  of  random  numbers 
for  output.  The  output  random  number  is  replaced  in  the  array  by  the  random 
number  used  to  select  its  position  in  the  array  [71]. 

This  random  number  generator  was  tested  on  various  numbers  of  dis¬ 
tinct  lesion  types.  The  results  of  using  the  random  number  to  generate  32,000 
random  numbers  between  1  and  12  are  illustrated  in  Figure  6.12.  Note  the 
relatively  equal  assignment  of  lesion  types. 

Results  of  lesion  assignment  simulations  A  total  of  16  simula¬ 
tions  were  performed  for  different  numbers  of  distinct  lesion  types.  The  average 
of  the  16  simulations  are  provided  in  Figure  6.13.  As  expected,  when  only  one 
distinct  lesion  type  is  used  96  duplicate  lesion  triads  result.  The  number  of 
duplicate  triads  reduce  as  the  number  of  distinct  lesion  types  are  increased.  A 
total  of  12  distinct  lesion  types  were  required  to  reduce  the  number  of  duplicate 
entries  to  less  than  2. 
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Figure  6.14:  The  pixel  coordinate  shift  to  provide  distinct  template  triads.  One 
of  twelve  available  shifts  are  randomly  chosen  for  lesion  coordinate  placement. 
These  shifts  provide  the  distinct  template  triads  required  for  the  Adaptive 
Template  tracking  algorithm  without  significantly  altering  the  orderliness  of 
the  lesion  template  array. 

Controlling  the  distinctness  of  a  lesion  To  implement  the  Adaptive  Tem¬ 
plate  method  some  means  of  controlling  the  ‘distinctiveness’  of  lesions  is  re¬ 
quired.  The  above  analysis  indicated  12  distinct  lesion  types  must  be  used  to 
provide  for  nonduplicated  lesion  triads  within  a  61  lesion  array.  This  required 
distinctiveness  is  obtained  by  varying  the  lesion  coordinate  by  one  or  two  pix¬ 
els.  This  is  easily  implemented  since  a  given  pixel  has  eight  neighboring  pixels 
a  single  pixel  away.  The  remaining  four  displacements  are  obtained  two  pixel 
displacements  away.  The  pixel  coordinate  shift  is  illustrated  in  Figure  6.14. 

The  Adaptive  Template  Algorithm  The  final  step  in  demonstrating  the 
efficacy  of  the  Adaptive  Template  method  was  developing  an  algorithm  to  place 
current  lesions  using  the  position  of  predecessor  lesions  as  a  location  reference. 
This  algorithm  was  developed  to  place  these  lesions  on  a  still  video  frame.  The 
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Figure  6.15:  Adaptive  Template  results 


algorithm  developed  to  plot  the  results  of  the  Adaptive  Template  method  uses 
the  first  template  to  plot  the  second;  the  first  and  second  to  plot  the  third; 
the  first,  second,  and  third  to  form  a  triad  and  plot  the  fourth  and  so  on.  The 


results  of  this  plot  are  provided  in  Figure  6.15. 


6.8  Lesion  Tracking  and  Image  Analyzer  software 

The  software  developed  to  test  the  different  concepts  and  methods  for  this 
chapter  are  found  in  programs  Lesion  Tracking  and  Image  Analyzer  (LETINA) 
and  RANDOM.  Structure  charts  for  these  programs  are  provided  in  Figures 
6.16  and  6.17.  Many  of  the  support  functions  for  program  LETINA  arc  identical 
to  the  support  functions  for  program  RETINA  described  in  previous  chapters. 
However,  variations  in  template  building  functions,  Lesion  Data  Base  building 
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functions,  and  the  actual  tracking  algorithms  are  different. 

The  lesion  tracking  algorithm  uses  a  similar  concept  to  that  described 
for  the  tracking  algorithm  using  vessels.  The  same  limited  exhaustive  search 
technique  is  used.  Figure  6.18  illustrates  how  this  limited  exhaustive  search 
was  implemented  for  lesion  tracking. 

Chapter  8  provides  results  of  additional  tests  performed  on  the  lesion 
tracking  concept. 


testdatabase 

reduce_bai_noise 

deconflict_templatc 

build_detachment_ 
data  base 


Figure  6.16:  Lesion  Tracking  and  Image  Analyzer  (LETIN^ 


ad  Image  Analyzer  (LETINA)  software 


Figure  6.17:  Program  RANDOM 
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Figure  6.18:  The  limited  exhaustive  search  using  lesion  templates 


Chapter  7 


Development  System  Instrumentation 

7.1  Overview 

This  chapter  describes  the  instrumentation  used  in  the  development  system. 
Where  pertinent,  a  description  is  provided  on  the  theory  of  equipment  opera¬ 
tion. 

The  development  system  configuration  is  illustrated  in  Figure  7.1  This 
configuration  operates  in  the  following  manner:  The  Olympus  mydriatic  fundus 
camera  provides  an  optical  image  of  the  retina  to  the  Panasonic  charge  coupled 
device  (CCD)  video  camera.  The  video  camera  converts  the  optical  image  into 
a  RS-170  video  signal.  This  signal  is  recorded  on  standard  VHS  format  video 
tape.  The  video  signal  is  also  routed  to  the  Matrox  PIP-1024  video  frame 
grabber.  The  frame  grabber  converts  the  RS-170  video  signal  into  a  512  x 
512  array  of  integers.  The  integer  magnitude  corresponds  to  the  gray  level 
value  of  an  individual  pixel  within  a  video  frame.  An  image  of  the  retina  is 
now  available  for  processing.  The  frame  grabber  is  hosted  on  a  Gateway  486 
personal  computer  (PC)  operating  at  33  MHz.  The  PC  executes  the  tracking 
algorithm  using  image  data  obtained  from  the  frame  grabber.  An  X  and  Y 
laser  correction  signal  is  derived  from  the  tracking  algorithm  and  provided  to  a 
pair  of  General  Scanning  AX-200  galvanometer  driver  amplifiers  via  the  Data 
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Figure  7.1:  Developmental  system  instrumentation 


Translation  control  board.  These  drivers  provide  the  necessary  signals  to  the 
General  Scanning  G108  optical  scanners  used  to  steer  the  laser  beam.  The 
surgical  laser  is  simulated  by  a  low  power  Helium  Neon  laser.  This  laser  is 
projected  through  a  neutral  density  filter  and  a  Uniblitz  LS6Z  laser  shutter. 
The  interface  between  the  PC  and  the  shutter  and  driver  amplifiers  is  provided 
by  the  RETINA  HW/SW  Interface  designed  by  the  author.  More  information 
is  now  provided  on  each  component. 
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7.2  The  Fundus  Camera 

The  fundus  camera  is  an  optical  camera  used  to  image  the  surface  of  the  retina. 
There  are  two  basic  types  of  fundus  cameras:  the  contact  type  and  the  non- 
contact  type.  With  a  contact  fundus  camera,  an  ophthalmoscopic  lens  touches 
the  patient’s  cornea.  This  type  of  fundus  camera  affords  a  field  of  view  up  to 
85  degrees.  The  noncontact  type  fundus  camera  does  not  use  a  contact  lens 
and  has  a  maximum  field  of  view  of  60  degrees[77]. 

For  this  study  an  Olympus  mydriatic  fundus  camera  model  GRC-W 
was  used  to  image  the  retina.  The  Olympus  GRC-W  requires  dilation  of  the 
patient’s  pupil  (mydriatic)  to  view  the  retina.  W’ith  the  pupil  dilated,  the  eye’s 
interior  is  illuminated  via  a  halogen  source  axially  aligned  with  the  camera's 
objective  lens.  Light  reflected  from  the  fundus  is  captured  by  the  objective 
lens  and  is  provided  to  an  eyepiece,  to  a  video  camera  port  equipped  with  a  C 
mount,  and  to  several  still  photography  ports. 

For  effective  fundus  imaging  the  optics  of  the  fundus  camera  must  obey 
the  Gullstrand  Principle.  This  principle  indicates  the  ray  bundles  used  for 
illumination  and  observation  must  be  separated  on  the  cornea  and  on  the  first 
surface  of  the  crystalline  lens.  This  is  accomplished  by  having  the  illumination 
source  form  a  bright,  luminous  ring  on  the  cornea.  This  light  ring  also  forms 
a  ring  on  the  anterior  portion  of  the  crystalline  lens.  The  a-ea  inside  the  ring 
is  called  the  corneal  window  and  it  is  used  for  observation  of  the  retinal  image 
[77].  This  prevents  the  bright  reflections  from  the  cornea  from  overwhelming 
the  dim  reflections  from  the  retinal  surface  [78]. 

A  real  aerial  image  of  the  retina  is  formed  by  the  ophthalmoscopic  lens. 
This  lens  forms  a  blurred  image  of  the  retina.  This  blurring  is  caused  by  bundles 
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on  me  retina 

Figure  7.2:  Noncontact  fundus  camera  optics  [77] 

of  rays  originating  from  different  points  on  the  retina  not  coming  to  a  sharp 
focus.  Each  of  the  bundles  has  an  individual  entrance  pupil  (IEP)  to  the  eye. 
Anterior  to  the  eye’s  pupil  plane  is  a  common  waist  of  all  IEPs  called  the  total 
entrance  pupil  (TEP).  Ideally  the  image  bundles’  IEPs  are  equal  and  concentric 
at  the  TEP.  This  is  possible  when  the  field  of  view  is  limited  to  values  below  60 
degrees  and  the  pupil  is  widely  dilated  [77].  Typically,  pupil  dilation  is  limited 
to  eight  millimeters  [79].  At  fields  of  view  greater  than  60  degrees  the  image 
bundle’s  IEP  cause  the  TEP  to  exceed  the  size  of  the  corneal  window  allotted 
for  imaging  [77]. 

The  curvature  of  the  retinal  surface  must  also  be  considered  to  obtain 
a  sharp  image.  An  additional  requirement  is  for  a  comfortable,  safe  level  of 
retinal  illumination.  This  illumination  requirement  calls  for  a  fundus  camera 
objective  lens  that  is  as  fast  (short  focal  length)  as  possible.  This  further 
complicates  the  curvature  problem  since  the  individual  entrance  pupils  for  the 
different  image  bundles  will  not  converge  to  a  single  point  as  required  by  the 
discussion  above.  These  conflicting  requirements  call  for  a  large  aperture  lens 
with  a  corresponding  small  depth  of  field.  The  lens  must  focus  both  paraxial 


!  52 


and  peripheral  fields  in  the  same  plane  despite  the  curvature  of  the  retinal 
surface.  Through  careful  choice  of  the  ophthalmoscopic  len’s  refractive  index, 
thickness,  and  curvature  the  individual  entrance  pupils  may  come  to  focus  at 
a  single  total  entrance  pupil[77]. 

Following  the  ophthalmoscopic  lens  is  a  set  of  field  lens  whose  purpose 
is  to  bend  all  of  the  individual  image  bundles  into  the  entrance  pupil  of  the 
recording  device  [77]. 

7.2.1  Fundus  Camera  Filters 

The  fundus  camera  may  be  used  with  various  inline  filters  to  enhance  certain 
portions  of  the  retinal  image.  Early  in  this  study  an  Olympus  GRCW-FGE 
green  filter  was  employed  to  enhance  the  retinal  vessel  network.  This  is  a  green 
bandpass  filter  with  a  maximum  transmittance  wavelength  of  530  nm.  The 
spectral  response  of  the  filter  was  obtained  with  a  Varian  2300  spectropho¬ 
tometer. 

A  568  nm  interference  filter  (Edmund  Scientific  J43,127)  was  also  used 
inline  with  the  fundus  camera  illumination  lamp.  This  filter  was  chosen  based 
on  the  work  of  Delori  et  al.  described  in  Chapter  3  of  this  document.  As 
expected,  vessel  contrast  enhancement  was  superior  to  that  obtained  with  the 
Olympus  filter.  The  spectral  response  of  this  filter  was  also  obtained  with 
the  Varian  2300  spectrophotometer.  Use  of  this  inline  interference  filter  also 
reduces  the  level  of  illumination  reaching  the  surface  of  the  patient’s  retina. 
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7.3  The  CCD  Video  Camera 

A  Panasonic  WV-CD20  charge  coupled  device  video  camera  was  connected  to 
the  Olympus  fundus  camera  via  a  standard  C  mount.  This  is  a  510  x  492  pixel 
camera  with  256  distinct  gray  levels.  The  camera  operates  at  a  standard  30 
frames  per  second.  The  camera  has  a  peak  sensitivity  of  .07  footcandles  at  530 
nm.  This  camera  provides  a  resolution  of  approximately  40  microns  on  the 
retinal  surface.  A  high  resolution  is  required  when  forming  lesions  on  the  order 
of  200  microns  and  to  obtain  a  detailed  retinal  diagnostic  map. 

7.4  The  Video  Frame  Grabber 

The  video  frame  grabber  chosen  for  the  development  system  was  the  Matrox 
PIP- 1024.  This  video  digitizer  provides  a  1024  x  1024  pixel  image  plane  which 
may  be  partitioned  in  various  configurations.  For  this  research  effort  the  frame 
grabber  was  partitioned  into  four  512  x  512  image  planes. 

The  frame  grabber’s  primary  purpose  is  to  provide  a  ‘snapshot’  of  eye 
movement.  Correction  information  for  the  Laser  Pointing  System  is  then  de¬ 
rived  from  the  still  image.  Many  other  frame  grabber  functions  provided  within 
the  PIP-EZ  MS  DOS  Software  Library  were  used  in  this  project.  The  interested 
reader  is  referred  to  [80]. 

7.4.1  Theory  of  Operation 

A  block  diagram  of  the  PIP-1024  is  provided  in  Figure  7.3.  The  PIP-1024 
is  equipped  with  three  separate  video  input  sources.  One  of  these  sources  is 
selected  via  software.  The  video  functions  of  the  PIP- 1024  are  driven  by  the 
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Figure  7.3:  The  Matrox  PIP-1024  video  frame  grabber  [81  j 


timing  from  the  selected  input  video  source.  A  stable  internal  video  source 
may  also  be  selected  [31]. 

The  video  source  signal  is  provided  to  a  sync  separator  where  the  image 
data  is  extracted.  The  data  is  then  passed  to  an  analog  to  digital  converter 
(ADC).  The  user  has  control  over  the  gain  and  offset  control  of  the  converter 
via  software.  This  allows  the  user  to  center  the  incoming  signal  in  any  portion 
of  the  ADC’s  range.  These  controls  are  similar  in  function  to  the  brightness 
and  contrast  controls  on  a  standard  monitor.  The  ADC  converts  the  analog 
video  signal  into  an  8  bit  digital  integer.  The  output  of  the  ADC  is  routed  to 
the  input  look-up  table  (ILUT)  [81]. 

The  ILUT  consists  of  8  separate  256-byte  maps.  The  ILUT  maps  the 
input  data  to  values  established  by  the  user.  As  previously  described,  the 
ILUT  has  been  employed  in  this  project  to  dramatically  increase  the  contrast 
of  the  retinal  vessels  against  the  retinal  background.  The  output  of  the  ILUT 
is  provided  to  the  frame  buffer  [81]. 

The  frame  buffer  is  a  1  Mbyte  random  access  memory  (RAM)  module. 
The  frame  buffer  may  be  accessed  for  reading  and  writing  using  a  Cartesian 
coordinate  system.  A  video  frame  is  stored  in  the  frame  buffer  when  frame 


grabbing  is  active.  For  this  project  the  snapshot  grabbing  mode  was  used.  In 
this  mode  a  single  frame  is  grabbed  and  stored  in  the  frame  buffer  [81]. 
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The  output  section  of  the  frame  grabber  consists  of  a  keyer  and  output 
look  up  tables  (OLUT).  The  video  keyer  allows  selection  of  either  the  incoming 
video  signal  for  output  or  the  output  may  be  obtained  from  the  frame  buffer. 
The  output  selected  is  routed  through  three  separate  OLUTs  (red.  green,  blue). 
These  three  OLUTs  may  be  adjusted  to  provide  a  pseudo-color  output  or  the 
green  OLUT  may  be  used  to  provide  monochrome  output.  The  frame  grabber 
was  used  in  the  monochrome  mode.  The  OLUTs  are  routed  to  Digital  to  Analog 
Converters  (DAC)  for  reformatting  into  a  video  output  [81]. 

7.5  Laser  Pointing  Hardware 

The  laser  pointing  hardware  consists  of  galvanometer  driver  amplifiers  and  op¬ 
tical  scanners  manufactured  by  General  Scanning  Incorporated.  The  AX-200 
driver  amplifiers  and  the  G-108  optical  scanners  were  chosen  for  the  develop¬ 
ment  system  due  to  their  ready  availability. 

7.5.1  Driver  Amplifiers 

A  separate  AX-200  driver  amplifier  is  provided  for  the  X  and  Y  optical  scanners. 
The  AX-200  is  a  solid-state  variable  output  impedance  amplifier.  The  variable 
output  impedance  permits  adjusting  the  amplifier  response  to  step  changes 
in  the  input.  The  AX-200  has  a  maximum  input  range  of  ±  1  vdc  with  a 
maximum  output  current  of  ±  1  amp  [74].  The  required  input  signal  to  direct 
the  laser  is  derived  from  the  tracking  algorithm  and  provided  via  the  RETINA 
Hardware/Software  Interface.  This  driver  provides  open  loop  control  of  the 


156 


optical  scanners  since  no  feedback  signal  is  provided  back  to  the  driver. 

7.5.2  Optical  Scanners 

The  optical  scanners  employed  in  the  development  system  are  a  set  of  G108 
galvanometers.  These  scanners  do  not  include  a  position  sensor  to  provide 
feedback  for  precise  positioning.  These  scanners  may  be  driven  sinusoidally 
at  up  to  1,275  Hz.  The  scanners  have  a  mechanical  peak-to-peak  rotation 
of  8  degrees  and  a  torque  rating  of  80  gm-cm.  A  mirror  is  attached  to  a 
separate  mount  attached  to  the  output  shaft  of  the  scanner.  Mirror  deflection 
is  proportional  to  drive  current  [75]. 

7.6  The  Laser  Shutter 

The  laser  shutter  chosen  for  the  development  system  is  a  self  contained  laser 
shutter  and  drive  unit  manufactured  by  Vincent  Associates.  The  shutter  is  a 
Uniblitz  LS6Z  driven  by  a  Uniblitz  D122  driver. 

The  shutter  has  a  6  mm  opening  for  the  laser.  The  shutter  blades 
are  coated  with  Aluminum  Silicon  Oxide  and  .001  Beryllium  Copper  which 
withstands  laser  energy  upto  5  W/mm 2  in  the  visible  wavelengths.  The  shutter 
rapidly  responds  to  input  signals.  The  shutter  only  requires  1.8  msec  to  open 
and  0.8  msec  to  close  [82]. 

The  Uniblitz  D122  driver  may  be  used  in  many  different  modes.  For 
this  application,  the  driver  was  configured  in  the  ‘Pulse  In’  mode.  This  mode 
provides  an  active  high  input  to  control  shutter  exposure  for  the  duration  of  a 
positive  pulse  applied  to  the  driver.  The  shutter  follows  the  pulse  applied  to  the 


driver.  The  driving  signal  was  provided  via  the  RETINA  HW/SW  Interface 
from  the  tracking  algorithm  [83]. 


157 


The  D122  driver  also  has  a  built-in  safety  feature  to  guard  against  power 
failure.  In  the  event  of  an  AC  power  failure  to  the  driver,  the  laser  shutter 
remains  closed  after  power  is  restored  until  manually  reset  by  the  user  [83]. 

7.7  The  Computer 

The  computer  selected  for  this  project  was  a  Gateway  486-33  computer.  This 
computer  serves  as  the  host  for  the  video  frame  grabber  hardware  board  and  the 
data  acquisition  and  control  board.  This  computer  is  equipped  with  an  Industry 
Standard  Architecture  (ISA)  bus.  This  bus  configuration  allows  compatibility 
with  the  hardware  boards.  The  ISA  architecture  contains  two  buses:  a  separate 
16  bit  input/output  bus  and  a  32  bit  memory  bus  [84]. 

7.7.1  Specifications 

The  Gateway  486-33  is  equipped  with  an  Intel  33  MHz  80486DX  central  pro¬ 
cessing  unit.  This  ‘chip’  is  equipped  with  a  8  KB  cache  controller  and  an 
on-chip  floating  point  unit.  The  input/output  bus  operates  at  8.3  MHz  [85]. 

7.7.2  Specifications 

The  tracking  algorithm  was  also  tested  on  a  Gateway  486-50  computer.  This 
computer  is  equipped  with  an  Intel  80486DX2/50  central  processing  unit.  This 
processor  is  also  equipped  with  a  8  KB  cache  controller  and  an  on-chip  floating 
point  unit.  The  input/output  bus  on  this  computer  also  operates  at  8.3  MHz 
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[86]. 

7.8  Data  Acquistion  and  Control  Hardware 

Considerable  information  is  transferred  into  and  out  of  the  tracking  algorithm. 
The  main  interface  between  the  tracking  algorithm  and  the  external  hard¬ 
ware  is  the  Data  Translation  DT2801A  analog  and  digital  input/output  (I/O) 
board.  This  board  contains  its  own  resident  microprocessor  and  software  li¬ 
brary  to  perform  data  transfer  operations.  The  board  is  configured  with  a 
programmable  analog  input  channel,  two  analog  output  channels,  and  a  16  bit 
digital  input/output  port.  A  block  diagram  of  the  board  is  provided  in  Figure 
7.4  [88]. 
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The  DT2801A  board  is  controlled  by  a  number  of  high  order  program¬ 
ming  languages.  For  this  project,  the  board  was  accessed  via  C  functions  res¬ 
ident  in  the  Data  Translation  PC  Lab  software  library.  This  library  is  linked 
with  the  compiled  tracking  algorithm  just  prior  to  program  execution.  Use  of 
software  version  V03.02  (or  later)  allows  full  compatibility  with  the  486  PC 
operating  at  33  and  50  MHz  [88]. 

7.9  The  RETINA  HW/SW  Interface 

The  RETINA  HW/SW  Interface  is  a  self  developed  hardware  ‘black  box’  to 
fully  integrate  all  PC  hosted  boards,  external  drivers,  and  peripheral  hardware. 
The  interface  also  simulates  signals  that  would  be  provided  by  the  Reflectance 
Based  Feedback  Control  System.  A  schematic  of  this  interface  is  provided  in 
Figure  7.5. 

Specifically,  the  RETINA  HW/SW  Interface  provides  the  following  func¬ 
tions: 

•  Provides  an  interface  between  the  Data  Translation  DT2801A  I/O  board 
and  all  external  circuitry.  The  DT2801 A  is  linked  to  the  RETINA  HW/SW 
Interface  circuit  board  via  a  50  conductor  ribbon  cable. 

•  Simulates  the  ‘Lesion  Complete’  signal  from  the  Reflectance  Based  Feed¬ 
back  Control  System  (RFS).  The  RFS  provides  a  signal  when  a  lesion 
has  reached  its  prescribed  size.  This  signal  is  simulated  via  a  pushbutton 
on  the  front  panel  of  the  RETINA  HW/SW  Interface.  When  depressed  a 
bounceless  pulse  provided  by  the  cross-coupled  NAND  gates  (7400)  sets 
a  JK  flip-flop  (7476).  A  status  indicator  on  the  front  panel  is  also  set. 
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Figure  7.5:  The  RETINA  HW/SW  Interface 


The  tracking  algorithm  reads  the  status  of  the  flip-flop  via  the  function 
check  Jesion_status. 
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•  Provides  a  ‘Patient  Status’  control.  This  control  is  also  provided  on  the 
front  panel  of  the  RETINA  HW/SW  Interface.  This  allows  the  patient  to 
halt  the  laser  surgery.  When  depressed  a  JK  flip-flop  is  set  as  previously 
described.  A  status  indicator  is  also  set.  The  tracking  algorithm  monitors 
the  status  of  the  flip-flop  via  the  function  check_patient_status.  When  the 
tracking  algorithm  detects  a  set  flip-flop  the  laser  shutter  is  closed  and 
the  laser  surgery  is  brought  to  an  orderly  halt. 

•  Provides  for  laser  shutter  control.  The  actual  laser  shutter  control  signal 
is  generated  within  the  tracking  algorithm  and  the  DT2801A  I/O  board. 
The  RETINA  HW/SW  Interface  routes  the  laser  control  signal  from  the 
DT280IA  to  the  Uniblitz  D122  shutter  driver. 

•  Provides  the  laser  correction  signals.  The  actual  laser  correction  signal 
is  derived  within  the  tracking  algorithm  and  output  via  the  DT2801A 
in  analog  form.  However,  the  peak-to-peak  swing  of  the  laser  correc¬ 
tion  signal  is  ±  10  volts.  The  maximum  allowable  input  to  the  AX-200 
galvanometer  driver  amplifiers  is  ±  1  volts.  The  RETINA  HW/SW  In¬ 
terface  provides  20  K  ohm  trim  potentiometers  to  reduce  the  signal  from 
the  DT2801A  for  AX-200  compatibility.  These  ‘trimmers’  are  also  used 
to  calibrate  the  Laser  Pointing  System. 

•  Provides  the  decoding  circuitry  necessary  to  illuminate  the  proper  light 
emitting  diode  (LED)  on  the  Left  and  Right  Fixation  Arrays.  Recall, 
that  each  field  of  view  data  is  stored  separately  within  the  Patient  File. 
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When  a  given  field  of  view’s  lesion  data  is  retrieved  for  treatment,  the 
field  of  view  designator  is  decoded  from  the  hashing  function  code.  This 
code  is  converted  into  the  addressing  necessary  to  drive  the  4  to  16  line 
demultiplexers  (74154).  The  output  of  the  demultiplexers  illuminate  the 
proper  LED  on  the  fixation  arrays.  For  example:  if  field  of  view  5  of  the 
left  eye  is  being  readied  for  treatment,  the  LED  corresponding  to  field  of 
view  5  in  the  right  eye  is  illuminated. 

7.9.1  The  Fixation  Device 

The  purpose  of  the  fixation  arrays  are  to  minimize  eye  movement  during  laser 
irradiation.  By  fixating  on  an  object  the  patient  is  able  to  minimize  movement 
in  the  conjugate  eye.  Fixation  array  design  assumptions  are  based  on  the  work 
of  Cornsweet  and  Steinman. 

Cornsweet  [87]  proposed  a  saccadic  correction  mechanism  for  fixation 
based  on  an  optimal  locus  centered  on  the  fovea.  This  optimal  locus  is  assumed 
to  be  the  origin  of  an  error-signal  system  guiding  corrective  eye  movements. 
Studies  have  shown  the  standard  deviation  of  a  fixating  eye  is  on  the  order  of  5 
minutes  of  arc.  Steinman ’s  work  demonstrated  that  the  optimal  locus  described 
by  Cornsweet  is  small  and  invariant  in  position  on  a  given  retina  [89]. 

Steinman  studied  various  fixation  device  variables  including  size,  color, 
and  luminance  using  subjects  experienced  in  maintaining  fixation.  The  follow¬ 
ing  findings  from  Cornsweet’s  and  Steinman’s  work  were  used  to  design  the 
fixation  targets  for  this  study: 


•  Saccadic  eye  movements  are  reduced  with  the  use  of  a  fixation  target. 


•  The  primary  stimulus  for  involuntary  saccadic  eye  movements  is  displace¬ 
ment  of  the  retinal  image  on  the  retina. 
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•  Retinal  drift  movements  are  the  result  of  instability  in  the  oculomotor 
system. 

•  Even  when  a  subject  tries  to  fixate  on  a  target  the  eyes  are  in  constant 
motion. 

•  The  dispersion  of  subject  fixation  increases  monotonically  with  target 
size. 

•  No  significant  differences  occur  in  fixation  stability  between  red,  blue, 
and  white  targets.  However,  all  subjects  tested  had  the  least  fixation 
dispersion  with  the  red  target  and  the  most  with  the  white  target. 

•  Increasing  target  luminance  reduces  the  variability  of  eye  position. 

Using  these  findings  as  a  guideline,  small  (.315  cm  diameter),  red  light 
emitting  diodes  biased  to  operate  at  a  safe  current  load  were  used  as  the  fixation 
target.  Sixteen  diodes  were  arranged  in  three  concentric  rings  as  illustrated 
in  Figure  7.6.  The  three  rings  had  axial  displacements  of  14.3  degrees,  28.6 
degrees,  and  44.6  degrees.  Each  ring  contained  six  LEDs  staggered  at  60  degree 
intervals.  A  target  LED  was  also  provided  in  the  center  of  the  array. 

The  fixation  background  was  painted  matte  (flat)  black.  Each  target 
background  was  mounted  to  a  sliding  bracket  that  was  fixed  to  the  fundus 
camera.  The  fixation  background  was  mounted  6.5  cm  from  the  corneal  sur¬ 
face  (7.2  cm  from  the  nodal  point  of  the  eye).  The  sliding  apparatus  allows 
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Figure  7.6:  The  fixation  device 

adjustment  for  different  displacements  between  the  eyes.  As  mentioned  previ¬ 
ously,  the  entire  array  is  controlled  via  the  tracking  algorithm  and  the  RETINA 
HW/SW  Interface. 


Chapter  8 


Development  System  Testing 

This  chapter  contains  test  descriptions,  results,  and  analysis  of  test  results  per¬ 
formed  on  the  developmental  Retinal  Tracking  Subsystem.  The  chapter  begins 
with  a  description  of  the  testing  configuration  followed  by  sections  on  tracking 
using  blood  vessel  template  methods,  timing  test,  and  lesion  template  methods. 
Background  information  is  provided  throughout  the  chapter  as  needed. 

8.1  Retinal  Tracking  Subsystem  testing  using  blood 
vessel  templates 

This  section  begins  with  an  equipment  configuration  description  used  for  test¬ 
ing  the  tracking  algorithm  based  on  blood  vessel  templates.  Following  the 
equipment  description,  the  testing  protocol  is  described  with  test  results  and 
analysis.  Algorithm  timing  results  are  then  provided  with  methods  of  reducing 
algorithm  execution  time. 

8.1.1  Test  system  configuration 

Figure  8.1  illustrates  the  test  configuration  used  to  test  the  tracking  algorithm 
based  on  blood  vessel  templates.  Note  that  the  Laser  Pointing  Subsystem  is  not 
part  of  this  test.  Results  of  testing  the  Laser  Pointing  Subsystem  are  provided 
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separately  in  this  chapter. 

Each  piece  of  equipment  was  described  in  Chapter  7.  In  this  configura¬ 
tion  the  Olympus  fundus  camera  is  equipped  with  a  Panasonic  WVCD-20  CCD 
video  camera  to  record  patient  eye  movement  on  the  Mitsubishi  video  cassette 
recorder  (VCR).  The  fundus  camera  is  also  equipped  with  a  568  nm  interference 
filter  (Edmund  Scientific  J43,127)  to  enhance  the  retinal  vessels.  The  record¬ 
ings  were  completed  on  several  subjects  over  a  period  spanning  10  months.  The 
recordings  formed  the  data  base  for  testing  the  tracking  algorithm. 

The  fundus  camera  was  not  used  during  tracking  algorithm  testing. 
The  Mitsubishi  VCR  provided  the  video  record  of  the  subject’s  retinal  eye 
movement  for  analysis.  The  VCR’s  video  output  was  provided  to  a  Panasonic 
monitor  for  viewing  and  to  the  Matrox  PIP-1024  frame  grabber  hosted  on 
the  Gateway  486-33  personal  computer  (PC).  The  video  output  of  the  frame 
grabber  was  displayed  on  a  second  Panasonic  monitor.  The  PC  also  hosted 
the  Data  Translation  DT2801A  acquisition  board.  The  RETINA  HW/SW 
Interface  provided  a  link  from  the  DT2801A  to  the  fixation  array  mounted  to 
the  fundus  camera  during  the  filming  experiment. 

8.1.2  Test  description 

This  section  describes  the  test  protocol  used  to  film  the  subjects’  retinal  eye 
movement  and  then  to  test  the  tracking  algorithm. 
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Test  subject  description 

Subjects  were  chosen  for  this  test  to  obtain  a  variety  of  age,  sex,  and  nationality. 
All  subjects  were  unpaid  volunteers  familiar  with  this  ongoing  research  effort. 
None  of  the  subjects  had  fixation  experience  except  for  subject  SBR. 

•  Subject  RCL  is  a  21  year  old  male  of  Indian  descent.  RCL’s  left  eye 
was  filmed  at  a  50  degree  fundus  camera  field  of  view  using  a  568  nm 
interference  filter. 

•  Subject  CSL  is  a  27  year  old  Caucasian  female.  CSL’s  left  eye  was  filmed 
at  a  50  degree  fundus  camera  field  of  view  using  a  568  nm  interference 
filter. 

•  Subject  SBR  is  a  34  year  old  Caucasian  male.  SBR’s  right  eye  was  filmed 
at  a  50  degree  fundus  camera  field  of  view  using  a  568  nm  interference 
filter.  SBR  had  participated  in  two  filming  exercises  with  fixation  prior 
to  this  filming  test. 

•  Subject  ICR  is  a  28  year  old  female.  ICR’s  right  eye  was  filmed  at  a  50 
degree  fundus  camera  field  of  view  using  a  568  nm  interference  filter. 

Test  protocol  for  filming  retinal  movement 

Each  subject  had  their  eye  dilated  30  minutes  prior  to  the  filming  experiment. 
The  subjects  were  briefed  on  test  procedure  and  the  use  of  test  results.  The 
subjects  were  asked  to  fixate  on  the  illuminated  light  emitting  diode  on  the 
fixation  array.  The  subject’s  were  told  to  close  their  eyes  if  any  discomfort 
was  experienced.  Total  exposure  to  the  fundus  camera  illumination  source  was 
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90  seconds  or  less  during  the  filming  of  the  fixation  sequence.  The  subject’s 
forehead  and  chin  were  stabilized  using  the  rests  provided  on  the  fundus  camera. 
Results  of  the  filming  were  recorded  on  standard  one-half  inch  VHS  format 
video  tape  with  the  VCR  set  to  standard  play  speed. 

Test  protocol  for  testing  the  tracking  algorithm 

Each  video  tape  of  retinal  eye  movement  was  processed  through  a  tesi  battery 
culminating  with  the  tracking  algorithm  test.  The  test  battery  consisted  of  the 
following  steps: 

1.  A  reference  image  was  arbitrarily  selected  from  the  video  sequence.  The 
reference  image  was  ‘snapped’  to  a  still  frame  from  the  video  tape  and 
stored  for  further  processing  on  the  PC’s  hard  disk. 

2.  The  position  of  the  optic  disk  centroid  was  then  determined  using  function 
find-optic-disk.  This  function  had  the  user  align  a  cursor  over  the  center 
of  the  optic  disk.  The  coordinate  of  the  optic  disk  center  was  retained 
for  later  use. 

3.  A  histogram  was  then  plotted  of  the  central  225  x  225  reference  image 
pixels.  This  histogram  provided  a  method  of  comparing  the  reference 
image  before  and  after  histogram  modification. 

4.  The  histogram  of  the  reference  image  was  then  modified  using  function 
mod_hisl.  This  function  was  described  earlier  in  this  document.  The 
histogram  thresholds  were  automatically  determined  by  function  deter- 
min<= -thresholds.  This  function  examined  the  histogram  for  the  first  and 
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last  non-zero  entry  to  determine  the  lower  and  upper  thresholds  prior  to 
histogram  modification.  The  new  upper  and  lower  histogram  thresholds 
were  preset  to  0  and  255.  A  measure  of  histogram  expansion  was  defined 
to  compare  histogram  modifications  as: 


new  upper  threshold  —  new  lower  threshold 

expansion  ratio  =  - - - - — — - - - - - r—r: 

current  upper  threshold  —  current  lower  threshold 

(8.1) 

5.  The  reference  image  with  histogram  modification  was  then  used  to  build 
a  two-dimensional  vessel  tracking  template  using  the  template  building 
process  described  in  Chapter  4. 

6.  After  the  template  was  built  a  Lesion  Data  Base  was  built  (assuming  a 
typical  treatment  for  diabetic  retinopathy)  using  the  process  described  in 
Chapter  4.  The  template  and  the  Lesion  Data  Base  were  both  stored  in 
a  ‘patient  file’  for  each  subject. 

7.  A  45  second  retinal  movement  sequence  was  then  used  to  test  the  tracking 
algorithm.  The  45  second  interval  corresponded  to  the  estimated  amount 
of  time  required  to  treat  a  single  retinal  field  of  view.  The  tracking  al¬ 
gorithm  was  tested  using  two  different  versions.  The  first  version  plotted 
the  movement  of  the  retina  by  tracking  and  plotting  the  position  of  the 
two-dimensional  template.  The  plotting  commands  artificially  slowed  the 
tracking  algorithm.  The  second  version  of  the  tracking  algorithm  had  all 
print  and  plot  commands  removed.  This  version  provided  code  timing 
status  of  the  tracking  algorithm. 
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8.1.3  Test  results 

The  results  of  the  test  battery  for  each  subject  are  provided  in  Figures  8.2.  8.3, 

8.4  and  8.5.  In  each  of  these  figures  a  reference  histogram  is  provided  before 
and  after  histogram  modification.  Also,  results  of  building  a  Lesion  Data  Base 
for  the  treatment  of  diabetic  retinopathy  is  provided  for  each  subject.  Finally,  a 
plot  of  the  position  of  the  first  horizontal  template  is  provided  over  a  45  second 
test  sequence.  Figure  8.6  provides  a  summary  of  results  for  the  test  battery. 
The  total  reference  template  response  provided  in  the  figure  is  the  sum  of  the 
six  individual  template  responses  multiplied  by  100. 

8.1.4  Analysis  of  test  results 

Overall,  the  template  building,  Lesion  Data  Base  building,  and  tracking  func¬ 
tions  performed  correctly  for  these  four  subject  sequences.  Subject  RCL  had 
excellent  contrast  between  the  retinal  vessels  and  retinal  background  even  with¬ 
out  histogram  modification.  Histogram  modification  further  enhanced  the  ves¬ 
sel  network.  The  histogram  expansion  ratio  was  2  for  subject  RCL.  Seven 
blinks  were  detected  during  the  test.  The  tracking  algorithm  correctly  regis¬ 
tered  a  loss  of  lock  condition,  re-established  lock  in  approximately  330  ms.  and 
continued  with  the  tracking  sequence.  The  tracking  sequence  was  halted  after 
the  45  second  period  using  the  Patient  Panic  switch  on  the  HW/SW  Interface. 

Subject  CSL  also  had  an  expansion  ratio  of  2.  The  span  of  the  refer¬ 
ence  image  histogram  prior  to  modification  was  0  to  155.  The  histogram  was 
truncated  to  0  to  127  prior  to  expansion  to  0  to  255.  CSL  blinked  once  during 
the  test.  The  tracking  algorithm  correctly  registered  a  loss  of  lock  condition, 
re-established  lock,  and  continued  with  the  tracking  sequence.  The  tracking 
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Figure  8.2:  Results  of  testing  subject  RCL.  Upper  left:  histogram  of  the  ref¬ 
erence  image.  Upper  right:  histogram  of  the  reference  image  after  histogram 
modification.  Lower  left:  results  of  Lesion  Data  Base  building.  Lower  right: 
results  of  the  tracking  sequence.  A  40  pixel  (approximately  1.86  mm)  reference 
line  is  provided. 
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Figure  8.3:  Results  of  testing  subject  CSL.  Upper  left:  histogram  of  the  ref¬ 
erence  image.  Upper  right:  histogram  of  the  reference  image  after  histogram 
modification.  Lower  left:  results  of  Lesion  Data  Base  building.  Lower  right: 
results  of  the  tracking  sequence.  A  40  pixel  (approximately  1.86  mm)  reference 
line  is  provided. 
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Figure  8-4:  Results  of  testing  subject  SBR.  Upper  left:  histogram  of  the  ref¬ 
erence  image.  Upper  right:  histogram  of  the  reference  image  after  histogram 
modification.  Lower  left:  results  of  Lesion  Data  Boise  building.  Lower  right: 
results  of  the  tracking  sequence.  A  40  pixel  (approximately  1.86  mm)  reference 
line  is  provided. 


Figure  8.5:  Results  of  testing  subject  ICR.  Upper  left:  histogram  of  the  ref¬ 
erence  image.  Upper  right:  histogram  of  the  reference  image  after  histogram 
modification.  Lower  left:  results  of  Lesion  Data  Base  building.  Lower  right: 
results  of  the  tracking  sequence.  A  40  pixel  (approximately  1.86  mm)  reference 
line  is  provided. 
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Subject 

RCL 

CSL 

SBR* 

ICR 

sex,  age,  eye 

m,  21, 1 

f,  27, 1 

m,  34,  r 

f,  28,  r 

histogram  expansion 
ratio 

2 

2** 

2** 

5 

lost  lock/ 
blink  detected 

7 

1 

0 

0 

total  reference 
template  response 

232 

55 

170 

143 

average  update 
time  (ms)*** 

220 

220 

220 

220 

total  tracking 
area  (sq  mm)**** 

4.39 

1.31 

.482 

***** 

*  Two  previous  fixation  experiences 

*  *  Reference  histogram  was  truncated  to  allow  histogram  expansion. 

***  40  x  40  search  area 

*  *  *  *  45  second  test  sequence 
*****  Algorithm  did  not  track 

Figure  8.6:  Summary  of  results  for  tracking  tests  performed  on  subjects  RCL, 
CSL,  SBR,  and  ICR. 


sequence  was  halted  after  the  45  second  period  using  the  Patient  Panic  switch 
on  the  HW/SW  Interface. 
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Subject  SBR’s  histogram  (0  to  140)  was  also  truncated  to  allow  a  his¬ 
togram  expansion  ratio  of  2.  Note  the  small  movement  of  SBR’s  retina  during 
the  45  second  sequence.  Recall  that  subject  SBR  had  completed  two  filming 
sequences  prior  to  this  filming.  The  ‘tight’  movement  pattern  may  indicate  a 
benefit  to  having  patients  practice  fixation  prior  to  a  treatment  session.  The 
tracking  sequence  was  halted  after  the  45  second  period  using  the  Patient  Panic 
switch  on  the  HW/SW  Interface. 

Subject  ICR  sequence  ‘stress’  tested  the  tracking  algorithm.  ICR  had  a 
difficult  time  maintaining  fixation  during  the  filming  sequence  due  to  discomfort 
with  the  retinal  field  illumination  source.  The  field  illumination  was  reduced 
to  comfortable  level.  The  reduction  in  field  intensity  dramatically  reduced  the 
contrast  of  the  retinal  vessels  against  the  retinal  background.  Histogram  mod¬ 
ification  did  not  significantly  improve  the  situation;  nevertheless,  I  elected  to 
test  the  tracking  algorithm  with  this  sequence  to  see  how  the  algorithm  would 
respond.  Several  problems  were  encountered.  First,  the  template  building  al¬ 
gorithm  correctly  chose  vessel  templates  as  required  for  3  of  the  6  templates. 
However,  the  3  other  template  locations  did  not  correspond  to  any  discernible 
retinal  feature.  This  was  due  to  the  absence  of  discernible  features  in  certain 
portions  of  the  image.  The  tracking  algorithm  also  had  difficulty  tracking  a 
specific  coordinate  on  the  retina.  A  loss  of  lock  condition  was  not  registered 
because  the  template  response  did  not  fall  outside  the  preset  loss  of  lock  con¬ 
ditions.  This  indicated  that  the  templates  chosen  did  not  differentiate  between 
a  correct  match  coordinate  and  an  incorrect  coordinate.  This  was  an  antici- 


pated  result  due  to  the  poor  template  selection  already  described.  The  video 
sequence  was  also  examined  frame-by-frame  to  determine  other  possible  rea¬ 
sons  for  this  situation.  It  was  found  that  retinal  features  used  to  track  retinal 
movement  were  not  visible  in  certain  video  frames  due  to  poor  image  contrast. 
Although  the  tracking  algorithm  would  not  be  used  under  such  conditions,  this 
test  provided  insight  into  the  limitations  of  the  tracker.  A  more  sensitive  CCD 
camera  would  allow  a  lower  retinal  field  illumination  and  hence  allow  tracking 
for  subject  ICR. 

Wide  variation  of  template  response 

During  the  testing  of  the  tracking  algorithm  a  wide  variation  of  the  individual 
tracking  templates  was  noted.  Individual  templates’  response  could  vary  from 
0.15  to  1.3  while  maintaining  lock.  Recall  from  Chapter  4  an  ideal  template 
response  is  1.0.  This  indicates  a  duplicate  response  to  the  reference  template. 
A  response  less  than  1.0  indicates  a  template  response  less  than  the  refer¬ 
ence  template  while  a  response  greater  than  1.0  indicates  a  response  greater 
than  the  refrrenc^  template.  The  templates  were  designed  to  be  normalized 
against  a  uniform  change  in  field  illumination  intensity  due  to  fluctuations  in 
the  illumination  lamp.  Thus  the  wide  variation  in  the  template  response  was 
a  puzzle.  A  non-uniform  field  illumination  pattern  was  hypothesized  as  the 
cause  of  this  variation.  To  test  this  theory,  the  fundus  camera  illumination 
source  was  projected  through  a  lens  (/  =  25  mm)  to  approximate  the  focusing 
power  of  the  eye  onto  a  Labsphere  0.5  reflectance  standard.  The  standard  had 
a  planar  surface.  The  reflectance  standard  used  a  barium-sulfate-based  coating 
called  Spectraflect  to  provide  a  uniform  reflectance  surface  over  a  wide  wave- 


1 


Figure  8.7:  The  fundus  camera  illumination  source  projected  onto  a  0.5  re¬ 
flectance  standard 

length  range  [90].  An  image  of  the  standard  was  'snapped'  and  a  histogram 
v  's  plotted  through  the  center  of  the  standard  using  function  row -histogram. 

The  results  are  provided  in  Figure  8.7.  Note  the  w'ide  variation  of  field 
intensity  as  a  function  of  radial  position  within  the  image.  The  variation  in 
gray  level  from  the  edge  of  the  image  to  the  center  of  the  image  is  50  gray 
levels.  A  similar  variation  pattern  occurs  in  actual  retinal  images.  This  vari¬ 
ation  produces  a  position  dependent  template  response  and  explains  the  wide 
variation  of  measured  template  response. 

Timing  Tests 

The  tracking  algorithm  was  timed  using  the  function  print-time.  This  function 
derives  a  time  ‘hack’  from  the  computer  clock  and  prints  time  to  the  nearest 
millisecond. 
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The  tracking  algc,:thm  was  hosted  on  a  486  PC  operating  at  33  MHz. 
With  this  host  the  algorithm  required  330  ms  to  establish  initial  lock.  Once 
initial  lock  was  established  a  position  update  was  provided  every  290  ms.  A 
portion  of  this  increment  (55  ms  on  average)  was  required  to  ‘snap'  a  still 
frame  and  165  ms  was  required  to  transfer  data  from  the  frame  grabber  to  the 
compute:  and  compute  a  position  update.  Approximately  one-third  of  the  165 
ms  was  used  transferring  the  image  data  from  the  frame  grabber  to  the  PC.  The 
remaining  two-thirds  were  used  to  calculate  the  new  position  update.  Finally, 
55  ms  was  required  to  recheck  the  laser  position  after  laser  position  update. 
Most  of  this  55  ms  is  used  to  ‘snap’  a  still  frame.  This  timing  distribution  is 
illustrated  in  Figure  8.8.  The  algorithm  was  also  tested  on  a  486  PC  operating 
at  50  MHz.  With  this  host  initial  lock  was  provided  in  280  ms  and  a  position 
update  every  210  ms. 

To  further  reduce  processing  time  the  frame  grab  used  to  check  the 
terminal  laser  position  was  combined  with  the  frame  grab  for  the  subsequent 
position  update.  This  combined  two  frame  grabs  that  occurred  back-to-back. 
This  trimmed  55  ms  from  the  update  cycle.  Also  the  10  pixel  pad  added  to 
the  search  area  as  a  safety  pad  was  removed  such  that  the  search  area  was  28 
x  28  pixels.  This  represented  a  fifty  percent  decrease  in  data  transfer  from  the 
frame  grabber  to  the  host  computer  and  a  fifty  percent  decrease  in  total  pixel 
calculations.  These  time  reduction  steps  are  summarized  in  Figure  8.8.  The 
reduction  of  search  area  reduced  the  upper  velocity  tracking  capability  of  the 
tracking  algorithm  to  10  degrees  per  second  but  improved  the  capability  of  the 
tracking  algorithm  to  maintain  the  laser  within  a  given  target  radius.  These 
trade-offs  will  be  discussed  in  detail  in  Chapter  10. 
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Figure  8.8:  Position  update  timing  distribution 
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Memory  model  selection  and  testing 

When  using  a  Microsoft  C  compiler  a  memory  model  size  must  be  specified. 
These  models  include  the  Tiny-Model,  Small-Model,  Medium-Model.  Compact- 
Model,  Large-Model,  and  Huge-Model.  The  Tiny-Model  is  for  programs  which 
can  be  contained  in  a  single  memory  segment  of  64  kilobytes.  This  restric¬ 
tion  includes  both  code  and  data.  The  Small-Model  allows  64  kilobytes  for 
code  and  64  kilobytes  for  data.  The  Small-Model  programs  (along  with  the 
Tiny-Model)  are  faster  than  the  other  memory  model  programs  because  all 
memory  addressing  is  performed  within  a  given  memory  segment  using  near 
calls  (intrasegment)  [91]. 

The  Medium-Model  provides  a  trade-off  between  program  execution 
speed  and  program  size.  Data  is  limited  to  a  single  64  kilobyte  segment  while 
the  program  may  contain  multiple  segments.  Data  is  accessed  with  near  ad¬ 
dresses  while  code  is  accessed  using  far  (intersegment)  addresses  [91]. 

The  Compact-Model  provides  for  multiple  data  segments  and  a  single 
code  segment.  The  Large-Model  provides  for  multiple  code  and  data  segments 
and  the  Huge- Model  should  be  used  if  arrays  larger  than  64  kilobytes  are  re¬ 
quired  in  a  program  [91]. 

Program  RETINA  was  compiled  and  executed  using  the  Medium-Model. 
Also,  the  medium  memory  model  for  the  Matrox  PIP- 1024  frame  grabber  and 
the  Data  Translation  DT2801A  were  linked  to  the  program.  Based  on  the  in¬ 
formation  above,  the  tracking  algorithm  should  perform  faster  using  the  Small- 
Model  [91]. 

A  separate  program  called  main-trk.c  was  written  that  contained  only 
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the  tracking  algorithm.  Template  and  Lesion  Data  Base  functions  were  not 
included.  This  program  was  hosted  on  a  486  PC  operating  at  33  MHz.  This 
change  of  memory  model  ‘trimmed’  approximately  five  percent  from  the  update 
time  of  the  Medium-Model.  Additional  methods  of  reducing  the  update  time 
are  presented  in  Chapter  9. 

8.2  Laser  Pointing  Subsystem  testing 

8.2.1  Testing  the  Laser  Pointing  Subsystem  with  simulated  retinal 
movement 

The  Laser  Pointing  Subsystem  was  tested  using  the  test  configuration  illus¬ 
trated  in  Figure  8.9.  This  is  the  same  configuration  illustrated  in  Figure  8.1 
with  the  addition  of  the  Laser  Pointing  Subsystem  described  in  Chapter  5.  A 
35  mm  slide  projector  was  used  to  project  a  retinal  image  onto  a  14  cm  diame¬ 
ter  circular  silicon  wafer.  The  silicon  wafer  served  as  a  light  weight,  ultra-flat, 
rigid  mirror.  The  wafer  was  attached  to  a  General  Scanning  G330  galvanome¬ 
ter  driven  by  a  General  Scanning  AX-200  driver  amplifier  via  an  aluminum 
coupling  post.  The  total  moment  of  inertia  for  the  mirror  and  post  assembly 
was  216.3  gm  —  cm2.  The  armature  inertia  of  the  G330  galvanometer  was  4 
gm  —  cm2.  The  ratio  of  the  two  were  54.1  which  allowed  the  mirror  to  be  used 
at  13  percent  (estimated  from  the  frequency  response  derating  curve  provided 
with  the  galvanometer)  of  the  galvanometer’s  unloaded  frequency  response  or 
16.9  Hz. 

A  1  Hz  triangular  wave  was  provided  to  the  AX-200  driver  amplifier 
controlling  the  G330  optical  scanner.  The  triangular  wave  amplitude  was  varied 
to  provide  equivalent  retinal  velocities  of  1.0,  2.0  3.2,  5.0,  6.4,  10.0,  12.8  and 
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Figure  8.9:  Laser  Pointing  Subsystem  test  configuration.  A  retinal  image  is 
projected  onto  a  test  screen  via  a  silicon  wafer  acting  as  a  mirror.  The  mirror 
is  attached  to  a  G330  optical  scanner  via  an  aluminum  coupling  post.  The 
mirror  is  deflected  to  provide  equivalent  retinal  velocities  of  1.0,  2.0,  3.2,  5.0, 
6.4,  10.0,  12.8,  and  16.0  degrees  per  second.  The  CCD  provides  video  of  the 
moving  ‘retina’  to  the  tracking  algorithm  which  calculates  laser  position  up¬ 
dates.  The  position  updates  drive  the  G108  optical  scanners  via  the  AX200 
driver  amplifiers  to  maintain  the  laser  on  the  prescribed  retinal  coordinate. 
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Figure  8.10:  Results  of  testing  subject  RCL’s  retinal  image  at  a  retinal  ve¬ 
locity  of  12.8  degrees  per  second.  The  lower  trace  is  the  deflection  signal 
provided  to  the  AX-200  driver  amplifier  controlling  the  G330  image  deflec¬ 
tion  galvanometer.  The  upper  trace  is  the  correction  signal  calculated  by  the 
tracking  algorithm  and  provided  to  the  AX-200  driver  amplifier  controlling  the 
X  (horizontal)  mirror  of  the  Laser  Pointing  Subsystem.  Chart  speed  was  125 
mm  per  minute. 

16.0  degrees  per  second.  The  tracker  consistently  lost  lock  at  16.0  degrees  per 
second.  The  signal  provided  to  the  deflection  mirror  and  the  correction  signal 
provided  by  the  tracking  algorithm  to  the  X  galvanometer  were  plotted  with  a 
strip  chart  recorder  for  each  retinal  velocity. 

The  tracking  algorithm  was  first  tested  without  function  check  Jaser -position. 
This  test  demonstrated  the  capability  of  the  tracking  algorithm  to  correctly 
calculate  a  position  update.  Results  of  this  test  for  subject  RCL  at  a  retinal 
velocity  of  12.8  degrees  per  second  are  provided  in  Figure  8.10.  The  same  test 
was  then  accomplished  at  a  retinal  velocity  of  16  degrees  per  second.  The  track¬ 
ing  algorithm  lost  lock  at  this  velocity  due  to  retinal  features  going  outside  the 
search  area.  The  results  of  this  test  are  provided  in  Figure  8.11. 

The  tracking  algorithm  was  then  tested  with  function  check  Jaser.position. 
This  function  corrects  for  minor  misalignment  between  the  frame  grabber  co- 
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Figure  8.11:  Results  of  testing  subject  RCL’s  retinal  image  at  a  retinal  ve¬ 
locity  of  16.0  degrees  per  second.  The  lower  trace  is  the  deflection  signal 
provided  to  the  AX-200  driver  amplifier  controlling  the  G330  image  deflec¬ 
tion  galvanometer.  The  upper  trace  is  the  correction  signal  calculated  by  the 
tracking  algorithm  and  provided  to  the  AX-200  driver  amplifier  controlling  the 
X  (horizontal)  mirror  of  the  Laser  Pointing  Subsystem.  Chart  speed  was  125 
mm  per  minute. 

ordinate  system  and  the  Laser  Pointing  Subsystem  coordinate  system  as  de¬ 
scribed  in  Chapter  7.  The  results  are  provided  in  Figures  8.12,  8.13,  8.14,  8.15, 
8.16,  and  8.17  for  subjects  RCL,  CSL,  and  SBR  respectively.  Note  the  minor 
corrective  signals  in  the  upper  trace  in  each  of  these  figures  as  compared  to 
Figures  8.10  and  8.11.  During  these  tests,  the  tracking  algorithm  calculated 
laser  position  updates  and  projected  the  simulated  therapeutic  laser  (HeNe) 
onto  the  retinal  image.  Thus,  the  entire  system  was  demonstrated.  Results 
were  recorded  on  video  tape. 


8.2.2  Testing  the  Laser  Pointing  Subsystem  with  lesion  templates 

The  Laser  Pointing  Subsystem  was  also  tested  using  lesion  templates.  The 
same  equipment  configuration  used  to  test  vessel  templates  (reference  Figure 
8.9)  was  used  to  test  lesion  templates.  For  this  test  a  35  mm  slide  of  a  human 
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Figure  8.12:  Results  of  testing  subject  RCL’s  retinal  image  at  a  retinal  veloc¬ 
ity  of  12.8  degrees  per  second.  The  lower  trace  is  the  deflection  signal  pro¬ 
vided  to  the  AX-200  driver  amplifier  controlling  the  G330  image  deflection 
galvanometer.  The  upper  trace  is  the  correction  signal  calculated  by  the  track¬ 
ing  algorithm  and  provided  to  the  AX-200  driver  amplifier  controlling  the  X 
(horizontal)  mirror  of  the  Laser  Pointing  Subsystem.  Note  the  minor  corrective 
signals  provided  by  the  function  check  Jaser  .position.  Chart  speed  was  125  mm 
per  minute. 


Figure  8.13:  Results  of  testing  subject  RCL’s  retinal  image  at  a  retinal  veloc¬ 
ity  of  16.0  degrees  per  second.  The  lower  trace  is  the  deflection  signal  pro¬ 
vided  to  the  AX-200  driver  amplifier  controlling  the  G330  image  deflection 
galvanometer.  The  upper  trace  is  the  correction  signal  calculated  by  the  track¬ 
ing  algorithm  and  provided  to  the  AX-200  driver  amplifier  controlling  the  X 
(horizontal)  mirror  of  the  Laser  Pointing  Subsystem.  Note  the  minor  corrective 
signals  provided  by  the  function  checkJaser.position.  Chart  speed  was  125  mm 
per  minute. 
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Figure  8-14:  Results  of  testing  subject  CSL’s  retinal  image  at  a  retinal  veloc¬ 
ity  of  12.8  degrees  per  second.  The  lower  trace  is  the  deflection  signal  pro¬ 
vided  to  the  AX-200  driver  amplifier  controlling  the  G330  image  deflection 
galvanometer.  The  upper  trace  is  the  correction  signal  calculated  by  the  track¬ 
ing  algorithm  and  provided  to  the  AX-200  driver  amplifier  controlling  the  X 
(horizontal)  mirror  of  the  Laser  Pointing  Subsystem.  Note  the  minor  corrective 
signals  provided  by  the  function  check  Jaser .position.  Chart  speed  was  125  mm 
per  minute. 


Figure  8.15:  Results  of  testing  subject  CSL’s  retinal  image  at  a  retinal  veloc¬ 
ity  of  16.0  degrees  per  second.  The  lower  trace  is  the  deflection  signal  pro¬ 
vided  to  the  AX-200  driver  amplifier  controlling  the  G330  image  deflection 
galvanometer.  The  upper  trace  is  the  correction  signal  calculated  by  the  track¬ 
ing  algorithm  and  provided  to  the  AX-200  driver  amplifier  controlling  the  X 
(horizontal)  mirror  of  the  Laser  Pointing  Subsystem.  Note  the  minor  corrective 
signals  provided  by  the  function  check-iaser.position.  Chart  speed  was  125  mm 
per  minute. 
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Figure  8.16:  Results  of  testing  subject  SBR’s  retinal  image  at  a  retinal  veloc¬ 
ity  of  12.8  degrees  per  second.  The  lower  trace  is  the  deflection  signal  pro¬ 
vided  to  the  AX-200  driver  amplifier  controlling  the  G330  image  deflection 
galvanometer.  The  upper  trace  is  the  correction  signal  calculated  by  the  track¬ 
ing  algorithm  and  provided  to  the  AX-200  driver  amplifier  controlling  the  X 
(horizontal)  mirror  of  the  Laser  Pointing  Subsystem.  Note  the  minor  corrective 
signals  provided  by  the  function  check  Jaser -position.  Chart  speed  was  125  mm 
per  minute. 


Figure  8.17:  Results  of  testing  subject  SBR’s  retinal  image  at  a  retinal  veloc¬ 
ity  of  14.0  degrees  per  second.  The  lower  trace  is  the  deflection  signal  pro¬ 
vided  to  the  AX-200  driver  amplifier  controlling  the  G330  image  deflection 
galvanometer.  The  upper  trace  is  the  correction  signal  calculated  by  the  track¬ 
ing  algorithm  and  provided  to  the  AX-200  driver  amplifier  controlling  the  X 
(horizontal)  mirror  of  the  Laser  Pointing  Subsystem.  Note  the  minor  corrective 
signals  provided  by  the  function  check-laser-position.  Chart  speed  was  125  mm 
per  minute. 
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Figure  8.18:  Subject  DL.  This  subject  was  treated  for  diabetic  retinopathy 
using  an  argon  laser.  Note  the  irregular  shaped  lesions.  Histogram  modification 
was  performed  on  the  image  via  input  look-up  table  modification  to  increase 
contrast.  The  lesions  used  for  tracking  are  highlighted  by  three  black  cursors 
in  the  central  portion  of  the  image. 

retina  with  actual  argon  laser  lesions  was  tested.  This  color  35  mm  slide  of 
photocoagulative  treatment  for  diabetic  retinopathy  was  provided  by  Dr.  H. 
Grady  Rylander.  The  subject  was  designated  DL.  As  before,  the  images  were 
moved  at  equivalent  retinal  velocities.  A  maximum  retinal  velocity  of  6.7  de¬ 
grees  per  second  was  tested.  Higher  retinal  velocities  were  attempted  but  the 
tracker  lost  lock.  This  was  due  to  a  reference  template  response  of  only  9. 

A  photograph  of  the  retinal  image  is  provided  in  Figure  8.18.  The 
signal  provided  to  the  image  deflecting  mirror  and  the  correction  signal  to  the 
horizontal  mirror  driver  is  provided  in  Figure  8.19  for  a  retinal  velocity  of  6.7 
degrees  per  second. 


t 


Figure  8.19:  Results  of  testing  a  human  retinal  image  with  actual  argon  lesions 
using  the  lesion  template  tracking  algorithm  at  a  retinal  velocity  uf  of  6.7 
degrees  per  second.  The  lower  trace  is  the  deflection  signal  provided  to  the 
AX-200  driver  amplifier  controlling  the  G330  image  deflection  galvanometer. 
The  upper  trace  is  the  correction  signal  calculated  by  the  tracking  algorithm 
and  provided  to  the  AX-200  driver  amplifier  controlling  the  X  (horizontal) 
mirror  of  the  Laser  Pointing  Subsystem.  Note  the  minor  corrective  signals 
provided  by  the  function  checkJaser.position.  Chart  speed  was  125  mm  per 
minute. 
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8.2.3  Analysis  of  results 

The  following  observations  were  based  on  the  results  of  testing  the  Laser  Point¬ 
ing  Subsystem  with  the  vessel  template  tracking  algorithm  and  the  lesion  tem¬ 
plate  tracking  algorithm: 

•  The  tracking  algorithm  using  vessel  templates  successfully  tracked  retinal 
movements  up  to  16  degrees  per  second.  At  16  degrees  per  second  the  al¬ 
gorithm  lost  lock  for  subject  SBR  because  portions  of  the  two-dimensional 
vessel  template  went  outside  the  fundus  camera  field  of  view. 

•  The  tracking  algorithm  using  vessel  templates  and  a  28  x  28  pixel  search 
area  was  able  to  constrain  the  laser  within  a  100  micron  target  radius  for 
retinal  velocities  less  than  two  degrees  per  second. 

•  At  retinal  velocities  greater  than  two  degrees  per  second  but  less  than  ten 
degrees  per  second  the  tracking  algorithm  maintained  lock  on  the  moving 
retina.  Due  to  the  143  ms  position  update  processing  time,  the  position 
update  lagged  behind  the  intended  target  coordinate.  The  lag  was  pro¬ 
portional  to  the  retinal  velocity.  Figure  8.20  provides  a  representative 
sample  of  the  relationship  between  lag  and  retinal  velocity.  The  tracker 
lost  lock  at  retinal  velocities  greater  than  ten  degrees  per  second  because 
retinal  vessel  landmarks  passed  outside  the  28  x  28  pixel  search  area. 

•  Cycles  of  the  X  (horizontal)  mirror  correction  signal  for  a  subject  were 
similar  but  not  identical.  Slight  variations  in  the  correction  signal  over 
a  cycle  were  due  to  different  reference  frames  being  ‘snapped’  for  update 
calculations. 
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LPS  lag  (microns) 

Figure  8.20:  Laser  Position  Subsystem  lag  versus  retinal  velocity.  This  is  a 
representative  sample  of  the  relationship  between  retinal  velocity  and  the  lag 
of  the  Laser  Positioning  Subsystem  from  the  intended  coordinate.  The  lag  was 
measured  from  the  center  of  the  laser  spot  to  the  intended  coordinate.  The 
center  of  the  laser  spot  did  not  always  correspond  to  the  brightest  pixel  in  the 
laser  spot.  This  accounts  for  the  odd  shape  of  the  graph. 

•  Although  the  video  camera  was  capable  of  providing  30  frames  per  sec¬ 
ond,  only  seven  frames  were  examined  per  second  due  to  position  update 
processing  time  delay. 

•  Function  checkJaser.position  provided  slight  positional  updates  for  mi¬ 
nor  misalignments  between  the  frame  grabber  coordinate  system  and  the 
Laser  Pointing  Subsystem  coordinate  system.  This  was  evident  as  slight 
correcting  traces  in  the  X  (horizontal)  mirror  correction  signal. 

•  Precise  coalignment  of  the  Retinal  Tracking  Subsystem  with  the  Laser 
Pointing  Subsystem  is  essential  for  successful  tracking  and  laser  place¬ 


ment. 
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•  The  tracking  algorithm  using  lesion  templates  tracked  videotaped  circular 
lesions  on  the  rabbit  retina  quite  well;  however,  when  tested  on  actual 
human  lesions  the  tracking  algorithm  consistently  lost  lock.  The  function 
to  find  lesion  templates  findJesion.templates  was  adjusted  to  search  for 
circular  as  well  as  noncircular  lesions.  When  tested  against  the  retinal 
slide  for  subject  DL  the  lesion  templates  used  were  for  noncircular  lesions. 

•  The  tracking  algorithm  using  lesion  templates  successfully  tracked  retinal 
velocities  up  to  6.7  degrees  per  second.  The  poorer  response  of  the  lesion 
templates  was  attributed  to  low  values  of  total  template  response.  The 
total  template  response  for  vessel  templates  were  on  the  order  of  40.  The 
total  template  response  for  lesion  templates  was  4. 

This  concludes  the  chapter  on  testing  the  Developmental  System.  The 
next  chapter  describes  in  vivo  testing  and  results. 


Chapter  9 


In  vivo  Development  System  Testing  in 
Pigmented  Rabbits 

9.1  Overview 

This  chapter  details  the  in  vivo  testing  performed  with  the  Retinal  Tracking 
Subsystem  and  the  Laser  Pointing  Subsystem  on  pigmented  cross  bred  Cali¬ 
fornian  and  New  Zealand  rabbits.  The  chapter  begins  with  a  discussion  of  the 
optics  required  to  place  a  laser  on  a  specified  coordinate  within  the  fundus  cam¬ 
era  field  of  view.  Safety  considerations  are  then  discussed  in  detail.  Safe  levels 
of  maximum  permissable  exposure  are  derived  for  the  Laser  Pointing  Subsys¬ 
tem.  The  in  vivo  preparation  of  the  rabbit  is  then  discussed  with  the  equipment 
configuration.  The  chapter  concludes  with  testing  results  and  demonstrations 
of  panretinal  photocoagulation  treatment,  repair  of  retinal  breaks  or  tears,  and 
a  lesion  matrix  experiment. 


9.2  Optical  configuration  for  in  vivo  testing 

The  Laser  Pointing  Subsystem  was  described  in  detail  in  Chapter  5  of  this 
document.  This  subsystem  was  then  tested  in  Chapter  8  with  the  Retinal 
Tracking  Subsystem.  The  subsystems  were  tested  by  projecting  a  retinal  image 
moving  at  a  calibrated  retinal  velocity  onto  a  screen  and  testing  the  closed  loop 
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feedback  mechanism  of  the  tracking  algorithm.  To  test  this  closed  loop  feedback 
mechanism  in  vivo  requires  the  addition  of  a  plano-convex  lens  in  the  laser  path. 
The  purpose  of  the  lens  is  to  bring  the  laser  to  a  focus  at  the  pupil  such  that 
the  laser  passes  through  the  center  of  the  pupil.  This  allows  the  laser  access  to 
the  retina  without  modifying  the  optics  of  the  fundus  camera.  A  beam  splitter 
is  provided  between  the  fundus  camera  and  the  pupil  to  allow  coalignment  of 
the  fundus  camera  field  of  view  and  the  Laser  Pointing  Subsystem  field  of  view. 
A  neutral  density  filter  is  required  to  reduce  the  8  mW  HeNe  laser  output  to 
a  safe  level.  Reference  Figure  9.1.  This  optical  configuration  was  adapted 
from  similar  systems  designed  for  the  scanning  laser  ophthalmoscope  [31].  The 
optical  configuration  was  tested  on  a  model  eye  prior  to  in  vivo  testing.  The 
model  eye  was  constructed  from  a  31.75  mm  table  tennis  ball  which  had  the 
first  6.35  mm  removed.  A  biconvex  lens  (/  =  25.4  mm)  was  affixed  to  the 
front  of  the  ball  to  simulate  eye  optics.  Simulated  vessels  were  drawn  inside 
the  ball.  The  ball  was  attached  to  a  ring  stand  and  used  to  align  the  optical 
arrangement.  Reference  Figure  9.1. 


9.3  Safety  considerations  for  in  vivo  testing 

It  is  imperative  that  test  rabbits  be  protected  from  harmful  levels  of  laser 
irradiation.  American  National  Standards  (A NS) '  ZI36.1  provides  guidance  for 
the  safe  use  of  lasers  and  laser  systems.  Guida  nnvided  in  this  document 
was  used  to  derive  safe  levels  of  laser  irradiaimn  lor  use  in  in  vivo  testing. 
Safe  levels  are  specified  for  laser  use  based  on  laser  wavelength,  laser  viewing 
configuration,  laser  type,  and  exposure  duration.  It  was  assumed  that  safe 
viewing  levels  derived  for  humans  would  also  be  safe  for  rabbits.  For  this 
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Figure  9.1:  Left:  optical  configuration  for  in  vivo  testing  and  Right:  model  eye 
used  to  align  system  prior  to  in  vivo  testing. 
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experiment  an  eight  mW  HeNe  laser  operating  in  a  continuous  wave  mode 
at  633  nm  for  experiments  of  300  second  duration  was  used  to  simulate  a 
therapeutic  laser.  The  liser  was  projected  into  the  rabbit’s  eye  via  the  pupil. 
This  is  defined  as  intrabeam  viewing.  The  laser  output  was  reduced  to  3.5 
microwatts  at  the  rabbit’s  pupil  with  a  spot  size  of  1.0  mm  by  employing  a 
neutral  density  filter  of  optical  density  3  (OD3).  Optical  density  is  given  as 
[«]: 


optical  density  =  log(\ /transmittance) 


(9.1) 


or, 


transmittance  =  dennty  (9.2) 

Therefore,  an  OD3  neutral  density  filter  has  a  transmittance  of  0.001. 

Maximum  permissable  exposure  levels  for  the  in  vivo  experiment  were 
derived  in  the  following  way: 

ANSI  Z136.1  provides  an  equation  for  exposure  duration  for  a  continu¬ 
ous  wave  laser  operating  in  the  visible  region  as  [93]: 


T,  =  10  x  102O(A-°-55°)  (9.3) 

For  a  wavelength  of  633  nm  (0.633  microns)  T\  equals  457  seconds. 
This  value  of  exposure  duration  specifies  an  equation  for  maximum  permissable 
exposure  as  [93]: 


199 


MPE  =  1.8  t3/ 4  x  1(T3  J  cm~2  (9.4) 

For  an  experiment  duration  of  5  minutes  ( t  =  300  seconds)  MPE  is 
calculated  as: 

MPE  =  1.8  x  3003/4  x  10“3  J  cm~2  =  129.8  x  10"3  J  cm~ 2  (9.5) 

This  may  be  converted  to  W  cm~ 2  using  [93]: 

MPE  =  129.8  x  10'3  J  cm~2/ 300  s  =  433  x  10~6  W  cm'2  (9.6) 

The  beam  diameter  at  the  rabbit’s  pupil  was  measured  as  1.0  mm.  This 
equates  to  an  area  of  7.85  x  10~3  cm2.  To  maintain  a  safe  operating  condition 
the  laser  power  at  the  pupil  must  be  under  3.4  microwatts  by: 

433  x  1(T6  W  cm~2  x  7.85  x  10‘3  cm2  =  3.4  x  10“6  W  (9.7) 

This  safe  operating  condition  is  maintained  with  the  inline  neutral  den¬ 
sity  filter  with  optical  density  OD3. 

9.4  In  vivo  experimental  method 

The  rabbit  subjects  were  handled  in  accordance  with  Animal  Resource  Cen¬ 
ter  protocol  T02880906.  Humane  treatment  of  the  rabbits  was  of  paramount 
importance.  The  experimental  configuration  of  Figure  9.2  was  used.  A  photo¬ 
graph  of  the  optical  equipment  configuration  is  provided  in  Figure  9.3. 
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Figure  9.2:  In  vivo  experimental  configuration 


-  -  - 

Figure  9.3:  Equipment  configuration  for  in  vivo  tracking. 


A  pigmented  cross  bred  Californian  and  New  Zealand  rabbit  was  anaes¬ 
thetized  intramuscularly  with  a  combination  of  Ketamine  (ketamine  hydrochlo¬ 
ride  35  mg/kg)  and  Rompun  (xylazine  hydrochloride  5.9  mg/kg).  Following 
injection  the  rabbit’s  right  pupil  was  dilated  with  Mydriacyl  1%  (Tropicamide 
1%  ophthalmic  solution).  Several  drops  of  Alcaine  were  placed  in  the  eye  as 
a  local  anaesthetic.  A  speculum  (Sontec  Instruments  10-1000  Barraquer  wire 
speculum,  small)  was  then  inserted  to  hold  the  eyelids  in  an  open  position.  Su¬ 
tures  were  then  placed  in  the  four  recti  (superior,  inferior,  medial,  and  lateral) 
muscles  to  facilitate  retinal  movement.  The  rabbit  was  then  secured  to  an  ani¬ 
mal  platform  via  a  safety  belt.  A  0.9  percent  saline  drip  was  used  periodically 
to  irrigate  the  cornea  to  maintain  moistness.  The  depth  of  anaesthesia  w as 
checked  at  five  minute  intervals  using  the  toe  pinch  method.  Reference  Figure 
9.4.  Following  the  experiment  Ocumycin  salve  (Bacitracin  Zinc  and  Polymyxin 
B  Sulfate  ophthalmic  ointment)  was  applied  to  the  eye  under  the  lid.  Also,  1 


Figure  9.4:  Rabbit  preparation  for  in  vivo  experiments.  Note  the  speculum  in 
place.  A  suture  is  shown  in  the  medial  rectus  muscle. 

cc  of  Pen  BP-48  (Penicillin  G  Benzathine  and  Penicillin  G  Procaine  in  aqueous 
suspension  antibiotic)  was  injected  intramuscularly. 

With  the  rabbit  preparation  complete,  final  coalignment  of  the  Retinal 
Observation  Subsystem  and  the  Laser  Pointing  Subsystem  was  accomplished. 
This  step  is  the  most  critical  for  a  successful  in  vivo  experiment.  To  properly 
coalign  the  two  systems  the  Laser  Pointing  Subsystem  is  programmed  to  trace 
a  rectangular  pattern.  However,  the  subsystem’s  optics  are  adjusted  such  that 
the  rectangular  pattern  comes  to  a  point  focus  at  the  corneal  surface.  This 
point  focus  is  adjusted  such  that  it  passes  through  the  center  of  the  fundus 
camera  illuminating  ring.  This  method  allows  projection  of  the  rectangular 
pattern  onto  the  retina  without  interfering  with  the  fundus  camera’s  imaging 
optics.  Once  past  the  pupil  point  focus,  the  rectangular  pattern  rediverges 
and  forms  a  rectangular  pattern  on  the  retina.  Reference  Figure  9.5.  The 
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Figure  9.5:  Rectangular  laser  pattern  on  the  retina. 


rectangular  pattern  is  then  coaligned  with  the  identical  rectangular  coordinates 
of  the  frame  grabber.  This  alignment  process  is  illustrated  in  Figure  9.6. 

After  subsystem  coalignment  a  reference  image  of  the  rabbit’s  retina 
was  ‘snapped’.  The  recti  muscle  sutures  were  gently  pulled  to  ensure  the  optic 
disk  with  the  accompanying  retinal  vessels  were  in  the  fundus  camera  field  of 
view.  Histogram  modification  was  not  used  to  improve  image  contrast  since 
the  gray  levels  present  in  the  rabbit  retinal  image  covered  much  of  the  available 
gray  level  spectrum.  A  tracking  template  was  then  built  along  with  a  Lesion 
Data  Base  using  techniques  described  in  Chapter  8.  Results  of  these  steps  are 
provided  in  Figure  9.7. 
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Figure  9.6:  In  vivo  alignment  of  the  Laser  Pointing  Subsystem.  A  rectangular 
pattern  is  traced  by  the  Laser  Pointing  Subsystem.  The  rectangular  pattern  is 
brought  to  a  point  focus  at  the  rabbit’s  corneal  surface.  The  point  focus  is  ad¬ 
justed  such  that  it  passes  through  the  center  of  the  fundus  camera  illuminating 
ring.  Once  past  the  pupil  focus,  the  rectangular  pattern  rediverges  and  forms 
a  rectangular  pattern  on  the  retina.  The  rectangular  pattern  is  then  coaligned 
with  the  identical  rectangular  coordinates  of  the  frame  grabber. 


Figure  9.7:  Preparation  for  the  in  vivo  experiment.  Top:  reference  image  of  the 
rabbit  retina.  The  optic  disk  is  oblong  in  shape.  Retinal  vessels  are  confined  to 
regions  near  the  optic  disk.  Bottom  left:  results  of  building  a  tracking  template. 
Bottom  right:  results  of  building  a  Lesion  Data  Base. 
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Figure  9.8:  Plot  of  laser  position  during  a  four  minute  tracking  experiment 
conducted  on  an  anesthetized  cross  bred  pigmented  rabbit.  A  40  pixel  (ap¬ 
proximately  1.86  mm)  reference  line  is  provided. 

9.5  In  vivo  experimental  results 

The  Retinal  Observation  and  Tracking  System  was  successful  in  simultaneously 
tracking  slight  retinal  movement  in  an  anaesthetized  pigmented  rabbit  retina 
and  maintaining  a  laser  on  a  prescribed  target  coordinate.  Precise  coalign¬ 
ment  of  the  Retinal  Tracking  Subsystem  with  the  Laser  Pointing  Subsystem 
contributed  to  successful  tracking  and  laser  placement. 

To  demonstrate  the  in  vivo  tracking  ability  the  rabbit’s  retina  was  video 
taped  during  a  four  minute  tracking  sequence.  During  the  four  minute  test  the 
rabbit’s  retina  was  moved  using  the  recti  sutures.  The  plot  of  laser  position 
during  the  four  minute  test  is  provided  in  Figure  9.8.  Also,  a  video  summary 
of  the  target  area  over  the  four  minute  period  is  provided  in  Figure  9.9. 
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9.6  In  vivo  panretinal  photocoagulation  and  retinal  tear 
demonstration 

9.6.1  Objectives 

Since  the  Robotic  Laser  System  is  under  development  for  the  photocoagulative 
treatment  of  diabetic  retinopathy,  macular  degeneration,  and  retinal  break  or 
tear  repair;  demonstration  of  these  techniques  in  vivo  using  the  Retinal  Track¬ 
ing  and  Observation  System  would  prove  useful.  To  transition  from  tracking 
using  a  lower  power  HeNe  laser  to  an  argon  ophthalmic  laser  required  several 
modifications  to  the  Laser  Pointing  Subsystem.  This  section  begins  with  a 
description  of  these  required  modifications  followed  by  a  description  of  prelim¬ 
inary  testing  completed  prior  to  the  actual  in  vivo  experiment.  This  section 
concludes  with  a  description  of  the  protocol  used  to  demonstrate  panretinal 
photocoagulation  and  retinal  tear  treatment  in  vivo  and  the  results  of  this 
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demonstration. 

9.6.2  Equipment  Configuration 

The  ophthalmic  laser  used  initially  for  this  demonstration  was  a  Coherent  Sys¬ 
tem  900  Argon  Laser  Photocoagulator.  This  is  an  all  lines  argon  laser  rated  at 
5.0  watts.  The  actual  output  from  the  laser  fiber  was  approximately  1.4  watts. 
To  use  the  argon  laser  in  place  of  the  low  power  HeNe  laser  required  several 
modifications  to  the  laser  delivery  optics  of  the  Laser  Positioning  Subsystem. 
The  optics  of  the  delivery  system  were  adapted  from  those  developed  for  the 
scanning  laser  ophthalmoscope  [95].  The  modified  Laser  Positioning  Subsystem 
is  provided  in  Figure  9.10. 

Lenses  LI  and  L2  are  biconvex  lens  (/  =  12.7  mm)  configured  as  a 
beam  collimator.  These  are  required  since  the  fiber  delivered  beam  from  the 
photocoagulator  is  highly  divergent.  Lenses  L3  and  L4  are  plano-convex  lenses 
configured  as  a  laser  beam  expander  [94].  The  beam  expander  is  followed  by  an 
aperature  iris  to  reduce  beam  radius.  Lens  L5  has  the  dual  purpose  of  focusing 
the  expanded  beam  to  a  spot  and  collimating  the  laser  scan.  In  other  words, 
it  converts  the  angular  deflection  of  the  galvanometer  driven  mirrors  into  a 
beam  displacement  referenced  from  the  undeflected  beam  path  [95].  Lens  L6 
brings  the  laser  raster  to  a  point  focus  at  the  pupil  so  that  the  laser  can  pass 
through  the  center  of  the  illuminating  ring  from  the  fundus  camera.  Once  past 
the  pupil,  the  beam  scan  rediverges  such  that  any  retinal  coordinate  within 
the  fundus  camera  field  of  view  may  be  targeted  by  the  Laser  Positioning 
Subsystem.  The  beam  splitter  between  the  fundus  camera  and  the  rabbit’s  eye 
allows  coalignment  of  the  frame  grabber’s  image  plane  as  viewed  through  the 
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Figure  9.10:  Modifications  required  to  the  Laser  Positioning  Subsystem  for 
argon  laser  delivery  using  a  Coherent  System  900  Argon  Laser  Photocoagulator. 
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fundus  camera  with  the  coordinate  system  of  the  Laser  Positioning  Subsystem. 
A  removable  5  OD  argon  filter  is  mounted  between  the  beam  splitter  and  the 
fundus  camera  to  protect  the  CCD  camera  from  the  argon  beam.  The  filter  is 
removed  for  aligning  the  system  with  a  low  power  argon  beam.  After  alignment 
the  filter  is  dropped  in  place  to  protect  the  CCD  array  from  damage  and  avoid 
image  saturation.  The  filter  is  set  at  an  angle  to  cast  the  reflection  from  the 
filter  out  of  the  fundus  camera  field  of  view. 

This  optical  configuration  allowed  precise  beam  placement  on  the  rab¬ 
bit’s  retina.  However,  due  to  the  low  (1.4  W)  source  power  and  the  losses 
within  the  optical  system  the  power  delivered  to  the  cornea  was  only  20.8  mW. 
At  least  100  mW  was  desired.  The  major  source  of  power  loss  was  the  beam 
shaping  optics  (lenses  LI  to  L4  and  the  aperature  iris). 

To  remedy  this  situation  the  Coherent  System  900  was  replaced  with  a 
Coherent  Innova  100  Argon  Ion  Laser.  This  laser  can  deliver  up  to  20  watts 
of  argon  laser  power.  Also,  the  laser  may  be  delivered  without  a  fiber.  Use  of 
this  laser  allowed  the  removal  of  lenses  Ll  to  L4.  For  ease  of  laser  coupling  the 
plano-convex  lens  (Ll  in  Figure  9.11)  was  changed  to  /  =  750  mm.  Also,  a 
laser  shield  of  flat  black  (matte)  aluminum  foil  was  added  between  the  fundus 
camera  body  and  the  edge  of  the  beam  splitter.  This  shield  serves  the  dual 
purpose  of  increasing  the  quality  of  the  fundus  camera  image  and  blocking  the 
laser  beam  passing  through  the  beam  splitter.  The  modified  configuration  is 
shown  in  Figure  9.11. 
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Figure  9.11:  Modifications  to  the  optical  delivery  system  for  using  the  Innova 
100  Argon  Ion  Laser. 


9.6.3  Preliminary  Testing 

Prior  to  performing  a  panretinal  and  retinal  tear  repair  photocoagulation  demon¬ 
stration  preliminary  system  tests  were  performed  to  verify  system  operation. 
The  tests  included  a  computer  simulation  of  panretinal  photocoagulation  while 
tracking  actual  human  retinal  movement,  performing  panretinal  photocoagu¬ 
lation  on  a  stationary  paper  target  ‘retina1,  and  demonstrating  treatment  of 
retinal  tear  repair  on  a  stationary  paper  target  ‘retina1. 

To  provide  a  simulation  of  panretinal  photocoagulation  on  actual  hu¬ 
man  retinal  movement,  a  Lesion  Data  Base  for  the  treatment  of  panretinal 
photocoagulation  was  built  from  a  representative  frame  from  the  videotape  of 
subject  SBR  (34  year  old  Caucasian  male,  right  eye,  filmed  with  a  568  nm  in¬ 
terference  filter,  histogram  expansion  ratio  2).  A  tracking  template  was  also 
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Figure  9.12:  Panretinal  photocoagulation  simulation  on  human  retinal  move¬ 
ment.  Left:  desired  lesion  pattern  and  Right:  laser  position  plot  on  a  moving 
retinal  image.  If  the  target  retina  had  been  stationary,  the  right  plot  would  be 
an  exact  duplicate  of  the  left  illustration. 

built  using  procedures  previously  described  in  Chapter  4.  The  position  of  the 
laser  coordinate  was  plotted  on  one  of  the  frame  grabber’s  imaging  planes  dur¬ 
ing  the  tracking  sequence.  The  results  are  illustrated  in  Figure  9.12.  The  left 
illustration  is  the  desired  lesion  pattern  while  the  right  illustration  shows  the 
laser  plot.  The  right  plot  is  for  a  moving  retinal  image.  If  the  target  had  been 
stationary,  the  right  plot  would  be  an  exact  duplicate  of  the  left  illustration. 
The  laser  time  on  lesion  target  was  controlled  to  simulate  actual  irradiation 
time  for  a  100  mW  laser  power  at  the  cornea.  The  entire  panretinal  irradiation 
procedure  required  21.4  seconds. 

The  tracking  algorithm  was  slightly  modified  due  to  lessons  learned  from 
this  simulation.  Prior  to  the  simulation,  initial  tracker  lock  was  reestablished 
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every  time  a  new  lesion  coordinate  was  initiated.  This  was  not  required  since 
tracker  lock  is  not  tied  to  the  laser  target  coordinate.  This  slight  change  in  the 
tracking  algorithm  saved  considerable  time  in  the  panretinal  photocoagulation 
sequence  and  provided  a  more  accurate  lesion  placement. 

The  second  stage  of  the  preliminary  testing  involved  placing  actual  laser 
lesions  on  stationary  paper  target  ‘retinas1.  A  retinal  vessel  pattern  was  drawn 
on  a  white  thermal  paper  target  background.  A  Lesion  Data  Base  was  then 
built  for  the  treatment  of  diabetic  retinopathy  and  retinal  tear  repair.  The 
system  was  aligned  and  the  lesions  were  placed  as  prescribed  by  the  data  base. 
The  results  are  illustrated  in  Figure  9.13. 

These  simulation  exercises  on  thermal  paper  targets  allowed  fine  tuning 
of  the  in  vivo  experimental  procedure.  Specifically,  the  following  lessons  were 
learned: 

•  Templates  selected  for  retinal  tracking  should  not  be  close  to  an  intended 
lesion  target  coordinate.  If  the  laser  illuminates  the  retina  within  ap- 
proxii  lately  one  lesion  diameter  of  a  tracking  template  the  tracker  loses 
lock.  This  does  not  constrain  tracking  algorithm  operation  since  blood 
vessels  used  as  tracking  templates  are  not  viable  lesion  target  coordi¬ 
nates.  This  restriction  may  be  lifted  with  the  inclusion  of  additional 
argon  wavelength  blocking  filters  placed  between  the  fundus  camera  and 
the  CCD  video  imaging  camera.  Specifically,  an  Andover  Corporation 
OG-550  sharp-cut  filter  (550FG05-25)  [96]  would  serve  this  purpose.  This 
optical  filter  passes  wavelengths  above  550  nm  and  blocks  those  below  550 
nm.  The  characteristics  of  this  filter  was  measured  with  a  Hitachi  U3300 
spectrophotometer.  Results  are  provided  in  Figure  9.14. 
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Figure  9.13:  Results  of  photocoagulation  on  paper  retina  targets.  Top  left: 
desired  lesion  placement  for  panretinal  photocoagulation  as  viewed  through 
the  fundus  camera,  Top  right:  lesion  placement  shown  actual  size,  Bottom  left: 
desired  lesion  placement  for  retinal  tear  repair  as  viewed  through  the  fundus 
camera,  Bottom  right:  lesion  placement  shown  actual  size. 
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•  Templates  should  not  be  chosen  close  to  the  edge  of  the  fundus  camera 
field  of  view.  The  field  of  view  edge  provides  a  strong  template  response 
which  overwhelms  the  correct  template  response.  Proper  choice  of  tem¬ 
plate  thresholds  will  allow  discrimination  between  the  correct  template 
response  and  a  false  template  response. 

•  Careful  alignment  of  the  laser  delivery  optics  and  coalignment  with  the 
video  frame  grabber  imaging  plane  is  critical  for  correct  laser  place¬ 
ment.  Also,  care  should  be  taken  to  use  the  paraxial  regions  of  the  lenses 
within  the  laser  delivery  system  to  avoid  Seidel  (monochromatic)  aberra¬ 
tion  affects  [10].  If  marginal  portions  of  the  lenses  are  used  the  alignment 
rectangle  appears  distorted  and  lesions  will  not  be  placed  in  prescribed 
locations. 


9.6.4  In  vivo  demonstrations 

Based  upon  lessons  learned  from  testing  on  the  paper  targets,  several  slight 
modifications  were  made  to  the  system  optical  configuration.  Specifically,  the 
5  OD  filter  was  removed  from  the  optical  path  between  the  fundus  camera  and 
the  beam  splitter.  Although  the  filter  optically  worked  quite  well,  mechanically 
it  was  large  and  hampered  precise  alignment  of  the  system.  An  Andover  OG- 
550  filter  was  placed  inline  between  the  fundus  camera  and  the  CCD  video 
camera  as  shown  in  Figure  9.15. 


Figure  9.14:  Transmission  characteristics  of  an  Andover  Corporation  OG-550 
sharp-cut  filter  measured  with  a  Hitachi  U3300  spectrophotometer.  This  filter 
prevents  the  argon  laser  from  saturating  the  CCD  video  camera.  The  argon 
laser  when  used  in  the  all  lines  mode  provides  laser  output  at  specific  wave¬ 
lengths  between  454.4  nm  and  528.7  nm  [97]. 
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Figure  9.16:  In  vivo  test  results  for  diabetic  retinopathy  treatment.  A  Coherent 
Innova  100  laser  operating  at  514  nm  was  set  to  deliver  100  mW  to  the  cornea. 
This  power  setting  was  increased  to  145  mW  at  the  cornea  to  provide  visible 
lesions.  The  tracking  algorithm  maintained  an  average  irradiation  time  per 
lesion  of  267.3  ms.  These  parameters  were  chosen  to  provide  a  shallow  but 
visible  lesion  on  the  subject  retina.  Top:  retinal  map  illustrating  the  tracking 
template  and  the  intended  lesion  coordinates.  Bottom:  photograph  of  actual 
lesion  placement. 
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Figure  9.17  provides  results  of  the  in  vivo  test  for  treating  retinal  tear. 
Figure  9.17  top  provides  a  retinal  map  illustrating  the  tracking  template  and 
the  intended  lesion  coordinates.  Figure  9.17  bottom  is  a  photograph  of  the 
actual  lesion  placement  on  the  subject’s  retina  taken  immediately  following 
lesion  placement. 

Analysis  of  results 

The  in  vivo  treatment  for  diabetic  retinopathy  and  retinal  tear  repair  demon¬ 
strated  the  capability  of  the  Retinal  Tracking  and  Observation  System.  Many 
lessons  were  learned  from  these  two  demonstrations  including: 

•  Precision  placement  of  lesions  on  the  retina  for  the  treatment  of  eye  dis¬ 
eases  is  possible  with  the  prototype  system.  For  precise  lesion  placement 
the  Retinal  Observation  Subsystem  must  be  precisely  coaligned  with  the 
Laser  Positioning  Subsystem.  The  result  of  misalignment  is  evidenced  by 
a  displacement  between  desired  and  actual  lesion  placement  in  the  demon¬ 
stration  of  treatment  for  retinal  tear.  In  the  demonstration  lesions  were 
placed  in  correct  orientation  to  one  another  but  the  lesion  pattern  was 
displaced  from  its  intended  position.  Displacement  was  approximately 
1500  microns. 

•  The  prototype  system  is  very  effective  in  efficiently  placing  lesions  for 
treatment.  Each  lesion  required  (on  average)  267.3  ms  for  placement. 
To  determine  total  treatment  time  the  number  of  lesions  is  multiplied  by 
the  time  required  per  lesion.  For  example  the  treatment  of  retinal  tear 
repair  required  approximately  6  seconds  while  the  treatment  of  diabetic 
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Figure  9.17:  In  vivo  test  results  for  retinal  tear  repair  treatment.  A  Coherent 
Innova  100  laer  operating  at  514  nm  was  set  to  deliver  230  mW  to  the  cornea. 
The  tracking  algorithm  maintained  an  average  irradiation  time  per  lesion  of 
267.3  ms.  These  parameters  were  chosen  to  provide  a  shallow  but  visible  lesion 
on  the  subject  retina.  Top:  retinal  map  illustrating  the  tracking  template 
and  the  intended  lesion  coordinates.  Bottom:  photograph  of  actual  lesion 
placement. 
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retinopathy  required  approximately  16  seconds. 

•  A  Reflectance  Based  Feedback  Control  System  is  required  to  control  lesion 
depth  during  irradiation.  Although,  laser  parameters  were  fixed  for  a 
given  treatment  demonstration,  a  visible  lesion  was  not  always  rendered 
on  the  retina.  This  was  due  to  retinal  tissue  inhomogenieties  and  variation 
in  laser  approach  angle  in  reference  to  system  optics.  A  depth  control 
system  would  compensate  for  these  variations. 

•  A  wide  variation  in  laser  parameters  were  noted  among  subject  rabbits. 
The  retinal  tear  repair  demonstration  was  accomplished  first.  The  laser 
was  set  for  514  nm  and  230  mW  at  the  cornea.  These  parameters  provided 
visible  lesions  on  the  retina.  These  same  parameters  with  the  same  spot 
size  and  irradiation  time  was  then  used  on  the  same  rabbit  for  demon¬ 
strating  diabetic  retinopathy.  The  Retinal  Observation  and  Tracking  Sys¬ 
tem  correctly  placed  lesions  at  the  required  coordinates.  However,  retinal 
hemorrhage  occurred  during  the  formation  of  two  lesions.  Reference  Fig¬ 
ure  9.18.  The  hemorrhage  indicated  penetration  of  the  choroidal  layer. 
This  observation  again  attested  for  the  need  of  a  depth  control  system. 
The  diabetic  retinopathy  demonstration  was  successfully  demonstrated 
at  a  corneal  power  of  145  mW. 

•  To  maintain  alignment  during  the  duration  of  the  experiment  a  surgical 
table  is  needed  which  contains  both  the  fundus  camera  and  the  animal 
platform.  This  platform  must  has  the  capability  to  be  locked  into  a  sta¬ 
tionary  position.  For  both  the  retinal  tear  repair  demonstration  and  the 
diabetic  retinopathy  demonstration  the  fundus  camera  and  the  animal 
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Figure  9.18:  Retinal  hemmorhage  due  to  penetration  of  the  choroidal  retinal 
layer.  The  dark  mass  is  fresh  blood  from  the  third  lesion  formed.  The  blood 
is  obscuring  lesions  formed.  Note  blood  is  also  evident  on  two  lesions  in  the 
outer  lesion  ring. 

platform  were  on  separate  tables.  These  tables  were  on  moveable  casters 
for  equipment  portability.  Once  the  overall  system  was  aligned  the  slight¬ 
est  movement  would  cause  system  misalignment  and  misplaced  lesions. 


Lesion  matrix  experiment 

Several  in  vivo  experiments  were  accomplished  to  demonstrate  lesion  place¬ 
ment  in  a  matrix  pattern.  A  matrix  pattern  allowed  documentation  of  system 
characteristics.  Prior  to  accomplishing  this  experiment  a  slit  lamp  microscope 
table  with  lockable  casters  was  redesigned  to  support  the  fundus  camera  and 
the  animal  platform  on  the  same  stable  stand. 
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The  placement  of  lesions  was  modified  such  that  lesions  were  arranged 
in  a  matrix  pattern  with  750  micron  spacing  between  lesion  rows  and  columns. 
The  subject  rabbits  were  prepared  as  described  earlier  in  this  chapter.  The 
Innova  laser  was  set  for  514  nm  irradiation  and  various  corneal  powers. 

Results 

Figure  9.19  shows  the  desired  lesion  matrix  and  the  a~tua]  lesion  matrix  place¬ 
ment  achieved  with  the  Retinal  Tracking  and  Observation  System.  In  this  first 
lesion  experiment  corneal  power  was  set  for  155  mW.  The  laser  exposure  time 
of  267  ms  per  lesion  was  controlled  by  the  update  cycle  of  the  tracking  algo¬ 
rithm.  The  irradiating  laser  produced  severe  glare  along  the  field  of  view  edges 
which  often  obliterated  the  tracking  templates. 

Figure  9.20  shows  the  desired  lesion  matrix  and  the  actual  lesion  matrix 
placement  achieved  with  the  Retinal  Tracking  and  Observation  System  for  the 
second  lesion  experiment.  In  this  second  experiment  the  size  of  the  matrix 
was  reduced  to  avoid  some  of  the  edge  effects  experienced  during  the  first  ma¬ 
trix  experiment.  Two  OG-550  filters  were  mounted  inline  between  the  fundus 
camera  and  the  CCD  video  camera  to  further  reduce  glaring  effects  from  the 
irradiating  laser.  Corneal  power  was  set  for  165  mW.  Laser  exposure  time  was 
again  267  ms. 

Figure  9.21  shows  the  lesion  matrix  placement  achieved  with  the  Retinal 
Tracking  and  Observation  System  immediately  post-operative  (top)  and  ten 
minutes  post-operative  (bottom).  In  this  third  matrix  experiment  a  five  column 
four  row  matrix  was  placed  at  a  corneal  power  of  55  mW.  Row  1  had  a  laser 
exposure  time  of  267  ms,  Row  2  534  ms,  Row  3  801  ms,  and  Row  4  1068 
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Figure  9.19:  First  matrix  experiment  results.  I'op:  desired  lesion  placement  as 
viewed  through  the  fundus  camera.  Bottom:  actual  lesion  placement. 


Figure  9.20:  Second  matrix  experiment  results.  Top:  desired  lesion  placement 
as  viewed  through  the  fundus  camera.  Bottom:  actual  lesion  placement. 
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ms.  The  purpose  of  this  experiment  was  to  demonstrate  the  capability  of  the 
tracking  algorithm  to  maintain  the  laser  on  a  specific  lesion  coordinate  during 
multiple  position  updates. 

Summary  of  results 

Results  of  the  in  vivo  experiments  are  summarized  in  Figure  9.22. 

Analysis  of  results 

The  lesion  matrix  experiments  demonstrated  the  capability  to  precisely  place 
lesions  on  the  retina.  However,  several  necessary  system  improvements  were 
noted: 

•  The  requirement  for  precise  coalignment  between  the  Laser  Positioning 
Subsystem  and  the  Retinal  Observation  Subsystem  is  again  evident.  Al¬ 
though  the  modified  animal  platform  allowed  improved  alignment,  greater 
precision  is  desired.  This  can  be  achieved  by  mounting  the  Laser  Posi¬ 
tioning  Subsystem  to  the  same  stable  platform  as  the  Retinal  Observa¬ 
tion  Subsystem.  The  galvanometers  currently  available  do  not  allow  this 
modification. 

•  The  check  Jaser.position  function  could  correct  some  of  the  misalignment 
errors.  This  function  was  turned  off  during  the  in  vivo  experiments.  The 
prototype  system  is  not  fast  enough  to  simultaneously  use  this  function 
and  several  other  documentation  functions  required  during  in  vivo  testing. 

•  The  irradiating  laser  sometimes  produced  severe  glare  which  obliterated 
the  tracking  templates.  This  situation  can  be  remedied  with  the  inclusion 
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Figure  9.21:  Third  matrix  experiment  results.  Top:  A  lesion  matrix  of  five 
columns  and  4  rows  were  placed  such  that  Row  1  had  a  laser  exposure  time  of 
267  ms,  Row  2  534  ms,  Row  3  801  ms,  and  Row  4  1068  ms.  Corneal  power 
was  set  for  55  mW.  Note  the  variation  in  lesion  characteristics  for  a  given  row. 
(Top)  Immediately  post-operative  and  (Bottom)  ten  minute  post  operative. 
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Figure  9.22:  Summary  of  in  vivo  expe 
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igure  9.22:  Summary  of  in  vivo  experiments. 
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of  more  OG-550  filters  inline  between  the  fundus  camera  and  the  CCD. 
Also,  the  use  of  a  anti-reflection  coating  on  the  plano-convex  lenses  will 
further  reduce  glare.  The  current  matching  connector  between  the  fundus 
camera  and  the  CCD  video  camera  only  allows  two  OG-550  filters.  A 
modified  design  is  under  investigation. 

•  Lesion  placement  errors  and  glaring  were  pronounced  at  the  edges  of  the 
image  in  lesion  experiment  1.  Also,  at  the  extreme  edge  of  the  field  of 
view  the  laser  did  not  reach  the  retina  because  of  physical  limitations 
imposed  by  the  size  of  the  Laser  Positioning  System  lenses.  To  remedy 
these  problems  the  software  was  modified  to  avoid  the  field  of  view  edges 
on  lesion  experiments  2  and  3. 

•  On  average  each  lesion  required  267  ms  to  form  on  matrix  experiments 
1  and  2.  To  complete  the  matrix  of  86  lesions  for  experiment  1  only 
required  23  seconds. 

In  vivo  lesion  tracking  experiment 

The  final  in  vivo  experiment  was  designed  to  demonstrate  the  capability  of  the 
system  to  track  using  lesion  templates.  In  this  experiment  a  triad  of  lesions 
were  first  placed  using  retinal  vessels  as  a  tracking  template.  The  lesion  triad 
was  then  used  as  a  tracking  template  to  place  another  lesion  triad. 

Results 

Figure  9.23  illustrates  the  result  of  this  experiment.  The  lesion  triad  on  the  left 
was  placed  using  vessel  templates  while  the  lesion  triad  on  the  right  was  placed 


Figure  9.23:  In  vivo  lesion  template  tracking.  The  lesion  triad  on  the  left  was 
placed  using  vessel  templates  while  the  lesion  triad  on  the  right  was  placed 
using  the  left  lesion  triad  as  a  template.  The  laser  was  set  for  514  nm  and  150 
mW  at  the  cornea.  The  mean  irradiation  time  for  the  lesions  in  the  left  triad 
was  267  ms  while  the  mean  irradiation  time  for  the  right  triad  was  330  ms.  The 
laser  fired  at  the  apex  of  the  right  triangle  but  did  not  leave  a  visible  lesion. 

using  the  left  lesion  triad  as  a  template.  This  experiment  demonstrated  the 
feasibility  of  using  lesions  as  templates  to  place  other  lesions  using  techniques 
described  in  Chapter  6. 


Chapter  10 


The  Real  Time  System 

10.1  Overview 

This  chapter  analyzes  equipment  requirements,  specifies  actual  hardware  com¬ 
ponents  and  costs,  and  a  provides  a  design  for  a  real  time  tracking  algorithm. 
The  chapter  begins  with  trade-off  analysis  of  interrelated  factors  which  influ¬ 
ence  the  performance  of  a  real  time  system.  This  analysis  forms  the  basis  for 
selection  of  actual  hardware  which  follows.  A  complete  real  time  system  is  then 
specified. 

10.2  Sensitivity  Analysis 

In  Chapter  2,  graphs  were  provided  which  illustrated  the  relationship  between 
retinal  eye  movement  velocities,  desired  laser  target  radius,  and  the  number  of 
position  updates  required  per  second.  This  analysis  will  now  be  revisited  in 
much  greater  detail  and  used  to  specify  real  time  system  components.  This 
section  begins  with  definitions  of  factors  influencing  the  performance  of  the 
Retinal  Tracking  Subsystem.  A  detailed  analysis  of  the  interrelationship  of 
these  factors  then  follows. 
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10.2.1  Factors  influencing  Retinal  Tracking  Subsystem  performance 

The  following  factors  influence  the  performance  of  the  Retinal  Tracking  Sub¬ 
system: 

•  Desired  laser  target  radius  is  defined  as  the  radius  of  a  circle  which 
contains  the  laser  spot’s  centroid  movement  as  illustrated  in  Figure  10.1. 
Forty  microns  was  arbitrarily  chosen  as  the  lower  target  radius  bound 
used  in  the  trade-off  analysis  because  this  is  the  approximate  pixel  length 
of  the  development  system.  The  upper  bound  was  chosen  to  be  520 
microns  because  this  is  close  to  the  desired  diameter  of  therapeutic  lesions. 
Ideally,  laser  target  radius  should  be  zero.  At  the  other  extreme,  as 
the  target  radius  becomes  greater  the  control  of  therapeutic  lesion  size 
degrades.  These  two  extremes  are  also  illustrated  in  Figure  10.1. 

•  Different  types  of  retinal  movement  were  discussed  in  Chapter  2.  Intu¬ 
itively,  there  is  a  direct  correlation  between  the  velocity  of  retinal  move¬ 
ment  and  the  required  performance  of  the  tracking  algorithm.  As  the 
retinal  velocity  increases  the  tracking  algorithm  must  provide  positional 
updates  at  a  faster  rate  to  maintain  a  given  laser  target  radius.  At  faster 
retinal  velocities,  the  tracker  may  be  able  to  maintain  lock  on  the  moving 
retina  but  the  updated  laser  position  will  lag  behind  the  intended  target 
coordinate.  By  employing  visual  fixation  retinal  velocities  may  be  the¬ 
oretically  reduced  to  micro-saccades  and  micronystagmus.  However,  for 
safety  purposes  the  real  time  system  will  be  designed  to  track  at  retinal 
velocities  up  to  50  degrees  per  second. 
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Figure  10.1:  Target  radius  of  the  laser.  Top:  Target  radius  is  defined  as  the 
radius  of  a  circle  which  contains  the  laser  spot’s  centroid  movement.  Bottom 
left:  Ideally  the  target  radius  should  be  zero.  Zero  radius  allows  laser  placement 
directly  on  target.  Bottom  right:  As  the  target  radius  becomes  larger  the 
control  of  therapeutic  lesion  size  degrades. 
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•  The  number  of  laser  position  updates  required  per  second  are  directly 
driven  by  the  desired  laser  target  radius  and  the  upper  bound  of  retinal 
velocity  that  the  algorithm  must  track.  As  the  target  radius  becomes 
smaller  the  number  of  position  updates  required  per  second  increases. 
Also,  as  the  upper  bound  of  retinal  velocity  increases  the  number  of 
position  updates  required  to  maintain  a  given  target  radius  increases. 

•  The  tracking  algorithm  uses  a  limited  exhaustive  search  to  derive  laser 
position  updates.  To  calculate  a  position  update  requires  a  finite  amount 
of  computer  processing  time.  Intuitively,  as  the  number  of  position  up¬ 
dates  per  second  increase  the  amount  of  retinal  movement  that  may  occur 
between  position  updates  decreases  for  a  fixed  retinal  velocity.  Hence,  the 
required  processing  time  for  a  given  position  update  also  decreases. 

10.2.2  System  specification  trade-offs 

To  clearly  demonstrate  the  interrelationship  and  trade-offs  between  the  fac¬ 
tors  influencing  tracking  algorithm  performance,  the  following  analysis  uses 
techniques  described  in  Chapter  4  for  different  retinal  velocities.  Arc  length 
calculations  are  used  to  determine  the  number  of  laser  position  updates  re¬ 
quired  per  second  for  a  given  laser  target  radius  at  a  given  retinal  velocity. 
The  number  of  laser  position  updates  required  per  second  may  also  be  used  to 
determine  the  total  number  of  template  calculations  required  per  second.  This 
analysis  is  independent  of  system  implementation.  A  specific  system  may  then 
be  analyzed  by  converting  the  total  number  of  template  calculations  required 
per  second  to  the  amount  of  time  required  by  the  system  processor  to  complete 
these  required  calculations. 
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Figure  10.2  illustrates  the  relationship  between  parameters  influencing 
tracking  algorithm  performance.  To  reduce  target  radius  for  a  given  retinal 
velocity  the  number  of  position  updates  must  be  increased  (upper  left).  An 
increase  in  the  position  update  rate  decreases  the  time  the  retina  may  move 
between  updates.  The  potential  search  area  size  decreases  as  the  position  up¬ 
date  rate  increases  (upper  right)  due  to  limited  eye  movement  between  updates. 
The  number  of  computations  required  per  second  increase  with  the  size  of  the 
search  area  (lower  left).  The  computations  required  per  second  decrease  as  the 
position  update  rate  is  increased  (lower  right).  This  reduction  is  due  to  the 
reduced  search  area  size  and  hence  fewer  computations  as  the  position  update 
rate  increases.  The  double  ordinate  graphs  of  Figure  10.3  (bottom)  ties  these 
interrelated  factors  together  for  various  retinal  velocities.  Required  position 
updates  per  second  is  provided  on  the  left  y  axis.  The  right  y  axis  provides 
the  total  number  of  template  calculations  required  per  second.  Both  y  axis 
share  the  desired  laser  target  radius  on  the  x  axis  (in  microns).  Program  sen- 
sitiv.c  was  used  to  compute  the  number  of  updates  required  per  second  and 
the  amount  of  processing  time  (for  a  486-33  PC)  required  to  calculate  a  po¬ 
sition  update  for  a  given  retinal  velocity  and  laser  target  radius.  The  results 
are  provided  in  Figure  10.3  for  system  independent  implementation  and  for 
implementation  on  a  486-33  in  Figure  10.4. 

10.2.3  Interpretation  and  analysis  of  results 

A  double  ordinate  graph  is  used  to  plot  the  interrelationship  of  factors  influ¬ 
encing  tracking  algorithm  performance.  Required  position  updates  per  second 
is  provided  on  the  left  y  axis.  The  right  y  axis  provides  the  total  number  of 
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Figure  10.2:  Relationship  between  parameters  influencing  tracking  algorithm 
performance.  To  reduce  target  radius  for  a  given  retinal  velocity  the  number 
of  position  updates  must  be  increased  (upper  left).  An  increase  in  the  position 
update  rate  decreases  the  time  the  retina  may  move  between  updates.  The  po¬ 
tential  search  area  size  decreases  as  the  position  update  rate  increases  (upper 
right)  due  to  limited  eye  movement  between  updates.  The  number  of  compu¬ 
tations  required  per  second  increase  with  the  size  of  the  search  area  (bottom 
left).  The  computations  required  per  second  decrease  as  the  position  update 
rate  is  increased  (bottom  right).  This  reduction  is  due  to  the  reduced  search 
area  size  and  hence  few  computations  as  the  position  update  rate  increases. 
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Figure  10.3:  Trade-off  analysis  results  based  upon  te 
second.  These  graphs  describe  the  interrelationship  be1 
the  performance  of  a  tracking  algorithm.  The  number 
(solid)  are  graphed  as  a  function  of  retinal  velocity 
radius.  The  number  of  template  calculations  required 
provided  on  the  right  ordinate.  The  operating  paran 
velocity  and  target  aradius  are  obtained  by  drawing  a 
desired  target  radius  on  the  appropriate  retinal  veloci 
position  updates  and  template  calculations  are  provi< 
the  vertical  line  and  the  respective  plots. 
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Trade-off  analysis  results  based  upon  template  calculations  per 
graphs  describe  the  interrelationship  between  factors  influencing 
ce  of  a  tracking  algorithm.  The  number  of  laser  position  updates 
iphed  as  a  function  of  retinal  velocity  and  desired  laser  target 
umber  of  template  calculations  required  per  second  (dashed)  are 
he  right  ordinate.  The  operating  parameters  for  a  given  retinal 
arget  aradius  are  obtained  by  drawing  a  vertical  line  up  from  the 
radius  on  the  appropriate  retinal  velocity  graph.  The  number  of 
ites  and  template  calculations  are  provided  at  the  intersection  of 
ne  and  the  respective  plots. 
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Figure  10.4:  Trade-off  analysis  rsults  based  on  an  Intel  33  M 
cessing  time.  These  graphs  describe  the  interrelationship  bet’ 
encing  the  performance  of  a  tracking  algorithm.  The  numbe 
updates  (solid)  are  graphed  as  a  function  of  retinal  velocity 
target  radius.  The  amount  of  processor  time  required  to  com 
calculations  required  per  second  (dashed)  are  provided  on  t 
The  processor  is  too  slow  for  a  given  retinal  velocity  and  las 
radius  combination  if  the  processor  requires  more  than  one 
the  template  calculations  required  every  second. 
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template  calculations  required  per  second.  Both  y  axis  share  the  desired  laser 
target  radius  on  the  x  axis  (in  microns).  A  separate  graph  is  provided  for  the 
following  retinal  velocities:  600  degrees  per  second,  300  degrees  per  second,  100 
degrees  per  second,  50  degrees  per  second,  10  degrees  per  second,  and  1  degree 
per  second.  A  common  graph  containing  all  retinal  velocities  is  too  ‘busy’  to 
be  useful. 

These  graphs  may  be  used  to  determine  the  number  of  position  updates 
required  per  second  and  the  accompanying  total  number  of  pixel  calculations 
required  per  second  for  a  given  laser  target  radius  for  a  given  retinal  velocity. 
For  example:  for  a  laser  target  radius  of  160  microns  and  a  retinal  velocity 
of  50  degrees  per  second,  90  position  updates  are  required  per  second.  These 
90  position  updates  require  approximately  44,000  total  template  calculations 
per  second  (see  Figure  10.3).  Figure  10.4  indicate  that  0.76  seconds  of  486- 
33  processing  time  is  required  to  compute  the  44,000  total  pixel  calculations. 
This  type  of  data  can  be  determined  for  any  combination  of  retinal  velocity 
and  desired  target  radius. 

These  graphs  may  also  be  used  to  examine  the  capability  of  an  existing 
system.  If  the  number  of  position  updates  a  given  system  can  complete  in  a 
second  is  known  its  tracking  capability  for  a  given  retinal  velocity  at  a  given 
target  radius  may  be  measured.  Several  trends  may  be  noted  from  the  graphs: 

•  To  obtain  laser  target  radius  less  than  120  microns  requires  a  substantial 
increase  in  position  updates  per  second. 

•  Only  a  slight  increase  in  position  updates  per  second  is  required  to  im¬ 
prove  laser  target  radius  from  500  to  200  microns. 
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•  At  faster  retinal  velocities  and  smaller  laser  target  radii  the  number  of 
template  calculations  required  per  second  exceed  the  processing  capacity 
of  the  computer.  This  is  most  evident  in  Figure  10.4  for  retinal  velocities 
of  600,  300,  and  100  degrees  per  second.  For  these  graphs  the  486-33 
requires  tens  of  seconds  of  processing  power  to  perform  the  number  of 
template  calculations  required  every  second.  Methods  of  mitigating  this 
situation  will  be  discussed  later  in  this  chapter. 

•  A  faster  processor  will  shift  the  required  processing  times  graph  (dashed 
line)  down. 

10.3  Real  Time  Equipment  Specification 

The  analysis  in  the  previous  section  may  be  used  to  determine  component 
requirements  for  the  real  time  Retinal  Tracking  Subsystem  and  Laser  Pointing 
Subsystem.  This  section  will  establish  these  requirements  and  then  describe  the 
components  required  to  construct  a  real  time  system.  This  section  concludes 
with  the  cost  of  the  real  time  system. 

10.3.1  Real  time  system  parameters 

The  speed  and  processing  power  of  the  real  time  system  may  be  determined  by 
specifying  an  upper  bound  for  retinal  velocity  and  desired  laser  target  radius. 
In  Chapter  2  retinal  velocity  parameters  were  described  in  detail.  If  patient 
visual  fixation  is  used  an  upper  bound  of  retinal  velocity  may  be  safely  specified 
at  50  degrees  per  second.  The  tracking  algorithm  will  register  a  loss  of  lock 
condition  should  retinal  velocity  exceed  this  upper  bound.  A  laser  target  radius 
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2  lesion  diameter  spacing 


Figure  10.5:  Resolution  of  adjacent  lesions.  A  laser  target  radius  of  100  microns 
as  a  lower  bound  for  lesions  2  lesion  diameters  apart  will  prevent  the  overlap 
of  adjacent  200  micron  lesions. 

of  100  microns  as  a  lower  bound  for  lesions  2  lesion  diameters  apart  will  prevent 
the  overlap  of  adjacent  200  micron  lesions.  In  a  worst  case  situation  at  this 
target  radius  two  adjacent  200  micron  lesions  would  just  touch.  Reference 
Figure  10.5. 

The  50  degree  per  second  graph  of  Figure  10.3  indicates  approximately 
150  position  updates  are  required  per  second.  Figure  10.4  indicates  that  a  486- 
33  processor  has  about  the  required  processing  power  to  perform  the  necessary 
calculations.  With  these  parameters  established  system  components  will  now 
be  described  for  the  real  time  system. 

To  transition  from  a  prototype  system  to  a  real  time  system  requires 
an  increase  in  the  number  of  position  updates  per  second.  An  increase  in 
position  updates  may  be  accomplished  by  using  a  faster  frame  rate  camera 
and  a  faster  processor.  The  number  of  position  updates  does  not  directly 
equate  to  a  required  camera  frame  rate.  The  camera  frame  rate  must  be  faster 
than  the  number  of  position  updates  required  per  second  to  allow  for  position 
update  calculation?  *o  occur  between  subsequent  images.  For  example:  in 
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the  prototype  system  a  camera  with  a  30  frame  per  second  rate  was  used 
;however,  only  7  frames  per  second  were  used.  More  efficient  use  of  available 
camera  frames  is  possible  if  position  updates  can  be  calculated  in  minimal 
time.  The  position  processing  time  may  be  minimized  by  hosting  the  tracking 
algorithm  on  a  faster  processor.  For  example:  the  current  prototype  system 
could  provide  10  position  updates  per  second  if  hosted  on  a  486-66  processor. 
To  further  reduce  position  update  processing  time  a  frame  grabber  with  an 
onboard  arithmetic  logic  unit  could  be  used. 

10.3.2  The  camera 

The  preceding  analysis  indicates  a  camera  with  a  frame  rate  of  at  least  150 
frames  per  second  is  required  to  provide  for  position  updates.  Ideally,  a  camera 
resolution  of  512  x  512  pixels  with  256  gray  levels  is  required.  A  video  camera 
with  this  resolution  and  frame  rate  combination  is  not  readily  available.  A  256 
x  256  pixel  camera  may  be  used;  however,  a  lens  system  between  the  fundus 
camera  and  the  video  camera  is  required  to  focus  the  central  portion  of  the 
fundus  camera  field  of  view  on  the  video  camera’s  active  sensing  array.  Dalsa 
Incorporated  of  Waterloo,  Ontario  manufactures  a  256  x  256  CCD  camera  that 
operates  at  up  to  200  frames  per  second.  This  camera,  the  Dalsa  CA-D1-0256, 
is  compatible  with  other  high  speed  imaging  equipment  to  be  discussed  [99]. 
Dalsa  application  engineers  indicate  the  sensitivity  of  the  camera  at  30  frames 
per  second  is  comparable  to  other  CCD  cameras  at  30  frames  per  second.  Due 
to  a  linear  decrease  in  integration  time  at  higher  camera  speeds,  the  image 
quality  will  also  degrade  linearly.  Some  of  this  image  degradation  may  be 
offset  with  input  look-up  table  modifications  in  the  frame  grabber. 
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10.3.3  The  frame  grabber 

The  desired  features  of  the  real  time  system  frame  grabber  is  compatibility 
with  the  high  speed  CCD  camera,  an  on-board  processor  to  speed  template 
calculations,  and  compatibility  with  a  personal  computer  host.  The  choice  of 
a  PC  host  helps  to  contain  overall  system  cost. 

Imaging  Technology  Incorporated  of  Woburn,  Massachuesetts  manufac¬ 
tures  an  entire  product  line  of  high-performance  modular  image  processing  sub¬ 
systems.  In  particular,  their  Series  150/151  modular  processors  provide  some 
of  the  required  capability  for  this  system.  The  Series  150/151  is  a  module 
family  to  perform  real  time  image  operations  and  image  analysis.  The  Retinal 
Tracking  Subsystem  would  require  the  following  Series  150/151  components: 
VSI-150  Variable  Scan  Interface,  FB-150  Frame  Buffer,  and  ALU-150  Pipeline 
Processor  [100]. 

The  VSI-150  allows  the  Series  150/151  system  interface  to  a  variety  of 
nonstandard  video  sensors.  This  feature  will  allow  use  of  the  Dalsa  high  speed 
CCD  video  camera.  The  VSI-150  is  also  equipped  with  a  programmable  input 
look-up  table  which  allows  real  time  histogram  modification  for  image  contrast 
enhancement.  Histogram  modification  techniques  were  described  in  Chapter  3 
of  this  document.  The  FB-150  contains  the  frame  storage  for  the  system.  It  is 
equipped  with  a  single  512  x  512  by  16  bit  frame  buffer  to  store  intermediate 
results  and  two  512  x  512  by  8  bit  frame  buffers.  The  ALU-150  has  pipelined 
architecture  which  allows  simple  operations  on  data  passing  through  the  module 
to  be  performed  in  minimal  time  [100]. 

Two  additional  system  requirements  are  the  interface  to  the  host  com¬ 
puter  and  the  ITI  system  software  library.  An  AT  to  VME  bus  interface  is 
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required  for  the  ITI  system  to  work  with  a  PC  host  computer.  This  interface 
may  be  ordered  as  an  option  with  the  system.  The  ITI  system  is  supported  by 
the  ITEX  150/151  software  library  of  image  processing  functions.  This  soft¬ 
ware  is  directly  compatible  with  Microsoft  C  and  contains  a  wide  variety  of 
image  acquisition  and  processing  functions  [100]. 

10.3.4  The  Computer 

Previous  analysis  indicates  the  486-33  processor  almost  has  enough  computa¬ 
tion  power  for  the  real  time  system  configuration.  A  prudent  choice  would  be 
a  486-50  processor  or  the  recently  released  486-66  processors.  Intel  has  just 
released  a  faster  processor,  the  Pentium.  This  processor  will  be  able  to  run 
all  the  software  developed  for  X86  processors  [114].  The  Pentium  theoretically 
has  a  peak  performances  rating  of  112  million  instructions  per  second  (mips) 
while  the  486-66  processor  is  rated  at  54  mips  [115].  More  computation  power 
will  allow  for  system  refinement  and  additional  features. 

10.3.5  Galvanometers  and  driver  amplifiers 

As  discussed  in  Chapter  5  of  this  document,  galvanometers  and  drivers  with 
feedback  control  are  required  in  the  real  time  system.  Also,  galvanometers  with 
MIL  STD  levels  of  reliability  will  provide  an  additional  measure  of  safety  to 
the  real  time  system.  A  pre-configured  X-Y  scan  head  package  is  further  rec¬ 
ommended.  These  products  are  available  from  General  Scanning  Incorporated. 
The  X-Y  scan  head  package  may  be  equipped  with  mirrors  specially  matched 
to  argon  wavelengths.  These  mirrors  have  98.5  percent  minimum  reflectance 
between  480  and  514  nm.  There  are  rated  at  100  Wjcm 2  for  100  ns  pulses 


(model  XY2026A).  The  AE1000  controller  board  provides  the  necessary  closed 
loop  control  for  the  X-Y  scan  head  package  [101]. 
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10.3.6  Data  acquisition  system 

The  Data  Translation  DT2801A  data  acquisition  board  described  in  Chapter 
7  of  this  document  has  the  required  capability  to  support  the  real  time  system 
configuration. 

10.3.7  Laser  Shutter 

The  Uniblitz  LS6Z2  shutter  and  the  D122  shutter  driver  described  in  Chapter 
7  of  this  document  has  the  required  capability  to  support  the  real  time  system 
configuration. 

10.3.8  System  Description 

Figure  10.6  provides  the  configuration  of  the  real  time  system.  The  intent  is  not 
to  specify  a  final  system  design  but  to  specify  a  design  to  demonstrate  that  the 
necessary  technology  now  exists  to  implement  a  real  time  tracking  algorithm. 

10.3.9  Real  time  system  cost 

The  system  configuration  illustrated  in  Figure  10.6  costs  26,955  dollars.  This 
is  a  breakdown  for  equipment  costs: 
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Figure  10.6:  The  real  time  system  configuration. 
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Description  Price 

Dalsa  CA-Dl-0256  CCD  video  camera  2550.00 

ITI  Series  150/151  Image  Processor  (88  price)  14485.00 

486-XX  computer  configured  with  an  AT  bus  3000.00 

X-Y  scan  head  with  closed  loop  controllers  4200.00 

DT2801A  acquisition  board  with  PC-LAB  software  1660.00 
Uniblitz  LS6Z2  shutter  and  D122  shutter  driver  710.00 

Microsoft  C  version  7  350.00 

Total  26955.00 


10.3.10  Technical  concerns 

Several  technical  concerns  exist  with  the  real  time  system  implementation. 
First,  at  faster  frame  rates  image  quality  degrades  due  to  a  shorter  camera 
element  integration  time.  The  rate  of  image  degradation  is  proportional  to 
camera  frame  rate.  Second,  to  study  a  real  time  system,  an  inexpensive  image 
storage  capability  for  nonstandard  frame  rates  is  required.  Finally,  although 
frame  grabbers  are  available  with  onboard  arithmetic  logic  units,  the  instruc¬ 
tions  sets  are  quite  limited.  Care  must  be  taken  to  select  a  frame  grabber  with 
the  required  instructions. 

This  completes  the  description  of  the  real  time  system.  Chapter  11 
describes  related  research  work  required  to  complete  the  design  of  the  Robotic 
Laser  System. 


Chapter  11 


Conclusions 

11.1  Conclusions 

This  research  demonstrated  the  feasibility  of  tracking  human  retinal  movement 
to  stabilize  laser  positioning  during  photocoagulation  therapy.  A  development 
system  was  built  and  tested.  This  system  incorporated  automatic  Lesion  Data 
Base  building,  template  building  using  blood  vessel  and  lesion  templates,  a 
tracking  algorithm  to  compensate  for  retinal  movement,  and  a  Laser  Pointing 
System.  Closed  loop  feedback  laser  control  using  the  tracking  algorithm  was 
demonstrated  with  the  development  system.  Also,  panretinal  treatment  for 
diabetic  retinopathy  and  the  repair  technique  for  treating  detached  tears  and 
breaks  were  demonstrated  in  vivo.  The  requirements  and  equipment  necessary 
to  implement  a  real  time  system  was  provided. 

A  real  time  system  is  required  for  clinical  use.  However,  several  areas 
of  research  need  further  investigation  prior  to  implementation  of  a  real  time 
system.  This  chapter  reviews  these  areas  of  further  research  followed  by  a 
summary  of  significant  findings  from  this  research  study.  The  chapter  concludes 
with  suggestions  for  using  results  of  this  research  effort  in  other  related  research 
efforts. 
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11.2  Future  Improvements  and  Further  Research 

11.2.1  In  vivo  testing  on  Macaca  mulatto  monkeys 

The  in  vivo  testing  on  pigmented  rabbits  provided  valuable  insights  on  system 
performance  and  limitations.  Although  the  tissue  properties  of  the  rabbit’s 
retina  is  similar  to  the  human’s,  its  physical  features  are  quite  different.  An  in 
vivo  test  on  a  human  similar  subject  would  prove  beneficial.  A  Macaca  mulatto 
(rhesus  monkey)  was  chosen  as  the  test  subject. 

Only  minor  modifications  are  required  to  the  prototype  system  for  the 
in  vivo  monkey  experiment.  The  subject  animal  will  be  oriented  such  that  the 
forehead  is  facing  the  fundus  camera  as  illustrated  in  Figure  11.1. 

A  cooperative  experiment  between  the  University  of  Texas  at  Austin's 
Biomedical  Engineering  Program,  the  Air  Force’s  Laser  Biophysics  Research 
Branch  at  Armstrong  Laboratories,  Brooks  Air  Force  Base,  San  Antonio,  Texas, 
and  the  United  States  Air  Force  Academy  (USAFA)  at  Colorado  Springs,  Col¬ 
orado  is  tentatively  planned  for  Summer  1993.  Due  to  the  high  cost  of  the 
animal  subject  ($1000),  only  one  experiment  is  planned  using  two  monkey  sub¬ 
jects.  This  experiment  will  demonstrate  the  treatment  of  diabetic  retinopathy 
in  one  eye  and  the  treatment  for  retinal  tears  in  the  fellow  eye  for  one  sub¬ 
ject  monkey.  The  other  monkey  will  be  used  to  demonstrate  lesion  matrix 
placement  as  a  feasibility  study  to  use  the  Retinal  Observation  and  Tracking 
System  in  conjunction  with  ultrashort  pulse  laser  tissue  studies.  Results  will  be 
presented  in  the  literature.  The  following  people  will  participate  in  this  joint 
experiment:  Major  Steven  F.  Barrett  (USAFA);  Dr.  A.J.  Welch  (UT),  Dr.  H. 
Grady  Rylander  (UT),  M.D.;  Captain  Cameron  Wright  (UT );  Dr.  (Lt  Col) 


Figure  11.1:  Equipment  configuration  for  in  vivo  Macaca  mulati 
Use  of  a  monkey  test  subject  requires  orientation  of  the  subjt 
that  the  forehead  is  facing  the  fundus  camera. 
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Figure  11.1:  Equipment  configuration  for  in  vivo  Macaca  mulatto  experiments. 
Use  of  a  monkey  test  subject  requires  orientation  of  the  subject’s  head  such 
that  the  forehead  is  facing  the  fundus  camera. 
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Mark  E.  Rogers  (AL),  Dr.  (Major)  Cynthia  Toth,  M.D.  (AL),  Dr.  Clarence 
Cain  (AL),  Dr.  (Capt)  Pat  Roach  (AL),  and  Lt.  Stein  (AL). 


11.2.2  Solid-state  laser  diode  therapeutic  laser 

Dr.  Maya  Jerath  used  an  argon  laser  (488  and  514  nm)  for  her  research  on  the 
Reflectance  Based  Feedback  Control  System.  This  has  been  the  therapeutic 
laser  of  choice  for  many  ophthalmological  based  treatments.  Although  this  laser 
has  proved  extremely  effective,  it  has  some  significant  disadvantages.  These 
disadvantages  include: 

•  The  laser  is  quite  large.  Although  the  laser  is  conveniently  delivered  via 
an  optical  fiber,  the  actual  lasing  equipment  is  housed  in  a  large  cabinet. 

•  A  3  to  5  watt  argon  laser  requires  an  external  cooling  system  to  maintain 
the  lasing  mechanism  at  a  non-destructive  temperature.  This  is  typically 
provided  by  a  water  cooling  loop.  This  loop  creates  a  significant  mess 
when  it  leaks  (experienced  first  hand). 

•  The  laser  requires  special  power  requirements.  For  example  the  Coherent 
System  900  Argon  Laser  Photocoagulator  used  in  this  study  required  208 
VAC,  3  phase,  30  amp  power.  The  Innova  100  Argon  Ion  Laser  required 
480  VAC,  3  phase,  60  amp  power. 

•  Since  the  all  lines  argon  laser  beam  has  several  wavelengths  it  is  difficult 
to  completely  filter  out  the  laser  with  a  single  filter.  To  protect  the 
imaging  camera  from  the  argon  laser  source,  Dr.  Jerath  used  a  chopper 
wheel.  This  mechanism  was  quite  effective;  however,  it  required  critical 
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alignment  and  a  feedback  speed  control  mechanism  on  the  chopper  wheel 
motor.  Use  of  a  single  wavelength  laser  source  can  reduce  the  complexity 
of  the  Reflectance  Based  Feedback  Control  System. 


Some  of  these  disadvantages  may  soon  be  of  historical  interest.  A  1.5  W 
argon  photocoagulator  using  forced  air  cooling  and  operating  at  120  VAC,  20 
amp  power  is  now  available  from  Coherent  Incorporated.  This  portable  system, 
designated  the  Ultima  2000  Argon  Laser  System,  has  been  FDA  approved  for 
all  photocoagulator  clinical  applications  [102]. 

Solid  state  laser  diodes 

Solid  state  laser  diodes  are  recent  technological  developments  which  may  rem¬ 
edy  the  disadvantages  of  the  argon  laser  source.  Solid  state  laser  diodes  offer  the 
advantages  of  compact  size,  long  operating  life,  reduced  power  requirements, 
and  air  cooling  [103].  However,  a  possible  disadvantage  is  pain  associated  with 
heating  of  the  choroid  and  a  narrow  therapeutic  power  window  [1], 

Solid  state  laser  diodes  consist  of  a  semiconductor  junction  similar  to 
those  found  in  a  rectifying  diode.  In  standard  Ge  and  Si  rectifying  diodes, 
energy  is  released  predominantly  as  heat  during  the  recombination  of  holes 
and  electrons  at  the  semiconductor  junction.  In  III-V  and  II- VI  compound 
materials,  energy  is  released  as  photons  with  energies  approximately  equal  to 
the  bandgap  of  the  material.  When  reflective  facets  are  included  on  these  diodes 
a  laser  diode  results.  The  facets  reflect  the  light  energy  back  and  forth  through 
the  semiconductor  junction  to  further  stimulate  emission.  When  the  current 
within  the  junction  reaches  a  certain  threshold,  internal  losses  are  overcome  by 
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the  stimulated  emission  and  a  laser  output  results.  At  lower  junction  currents 
the  laser  diode  acts  as  a  light  emitting  diode  [109]. 


Current  technology 

Currently  solid  state  laser  diodes  are  not  available  in  shorter  (close  to  argon 
wavelengths).  Shorter  wavelength  diodes  have  been  produced  by  increasing 
the  aluminum  content  of  InGaAlP  diodes.  This  increase  in  aluminum  has  the 
disadvantage  of  decreasing  the  diode’s  lifetime.  Recent  work  by  Masselink  and 
Zachau  of  IBM’s  Watson  Research  Center  have  produced  incoherent  emissions 
at  wavelengths  as  short  as  585  nm  at  room  temperature  [110,  111]  using  InGaP. 
Active  research  in  this  area  continues. 

Experimental  work  using  the  diode  laser 

Currently  solid  state  laser  diodes  are  readily  available  from  810  to  850  nm  at 
peak  power  of  10  watts  [108].  Sato  et  al.  have  compared  the  effects  of  such  a 
diode  (810  nm)  to  the  argon  (488  and  514  nm)  laser  for  retinal  endophotoco- 
agulation.  Sato  examined  the  blood-retinal  barrier  (BRB)  breakdown  within 
rabbit  eyes  using  magnetic  resonance  imaging.  He  hypothesized  that  the  pos¬ 
sible  adverse  effect  of  BRB  breakdown  related  to  retinal  photocoagulation  may 
be  different  for  the  diode  laser  as  compared  to  the  argon  laser.  Sato  suggests 
that  evidence  “supports  the  notion  that  BRB  breakdown  can  promote  the  de¬ 
velopment  of  proliferative  vitreoretinopathy  [103]”. 

Sato  tested  the  hypothesis  on  rabbits.  Clinically  similar  lesions  using  the 
diode  laser  were  placed  in  one  of  the  rabbits’  eye.  The  other  eye  received  lesions 
from  the  argon  laser.  The  lesions  and  degree  of  BRB  breakdown  was  then  ex- 
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amined  two  days  and  seven  days  after  treatment.  Sato  noted  that  two  days  after 
treatment  the  argon  treated  eyes  showed  greater  leakage  than  the  diode  treated 
eyes.  Furthermore,  the  diode  lesions  displayed  mild  cellular  displacement  in  the 
pigment  epithelium  and  extensive  thrombosis  of  the  choriocapillaris  and  larger 
choroidal  vessels.  The  argon  lesions  displayed  severe  cellular  displacement  in 
the  pigment  epithelium  and  fewer  occlusions  of  the  choriocapillaris  and  large 
choroidal  vessels.  Sato  cautioned  against  extrapolating  these  results  to  humans 
because  of  a  fully  developed  retinal  vasculature  in  the  human  eye  [103].  Related 
work  has  been  accomplished  by  various  researchers  [104,  105,  106,  107]. 

The  use  of  diode  lasers  especially  at  argon  similar  wavelengths  appear 
promising.  Additional  research  in  this  area  is  required. 

11.2.3  System  integration  study 

Dr.  A.J.  Welch,  Dr.  H.G.  Rylander  III,  Dr.  W.S.  Weinberg,  Dr.  M.S.  Markow, 
Dr.  Y.  Yang,  Dr.  S.  Ghaffari,  Dr.  A.J.  Seltzer,  Dr.  M.R.  Jerath,  and  the  au¬ 
thor  have  spent  considerable  time  developing  the  Robotic  Lasei  System  for  the 
treatment  of  ophthalmologic  diseases  and  disorders.  Their  related  research  ef¬ 
forts  have  included  studies  on  basic  laser  tissue  interactions,  theoretical  basis  of 
lesion  reflectance,  lesion  reflectance  feedback  daemons,  conceptual  design  of  the 
Robotic  Laser  System,  optical  tracking  mechanisms,  real  time  feedback  control 
systems,  reflectance  based  feedback  control  algorithms,  and  digital  tracking 
mechanisms.  Much  of  this  work  has  been  cited  in  this  document.  An  integra¬ 
tion  study  between  the  Reflectance  Based  Feedback  Control  System  and  the 
Retinal  Observation  and  Tracking  System  is  required.  The  goal  of  this  study 
would  be  the  design  and  implementation  of  the  complete  Robotic  Laser  System. 
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A  preliminary  discussion  of  an  integrated  system  is  provided  here. 

This  discussion  begins  with  a  listing  of  the  key  equipment  requirements 
for  the  Reflectance  Based  Feedback  Control  System  and  the  Retinal  Observa¬ 
tion  and  Tracking  System.  Alternative  methods  for  implementing  an  integrated 
system  are  then  presented  with  associated  advantages  and  disadvantages. 

Individual  system,  requirements 

This  section  describes  the  special  equipment  and  algorithmic  requirements  of 
the  Reflectance  Based  Feedback  Control  Subsystem  and  the  Retinal  Observa¬ 
tion  and  Tracking  Subsystem. 

Implementation  details  of  the  Reflectance  Based  Feedback  Control  Sub¬ 
system  are  provided  in  [4].  Careful  study  of  this  document  reveals  the  following 
equipment  requirements  for  this  subsystem: 

•  A  high  resolution  CCD  video  camera  is  required  to  image  the  forming 
lesion.  Jerath  used  a  512  x  512  pixel  camera.  The  camera  should  have 
the  capability  to  shut  off  automatic  gain  control  ( AGC).  The  AGC  feature 
adversely  affects  lesion  reflectance  measurements. 

•  The  lesion  imaging  CCD  video  camera  must  have  enough  pixel  resolution 
to  adequately  measure  central  reflectance.  Jerath ’s  system  monitored 
the  central  5  pixels  of  the  lesion  in  real  time.  These  5  pixels  represented 
0.05%  of  the  total  lesion  area. 

•  The  argon  source  must  be  completely  blocked  during  reflectance  mea¬ 
surements.  Any  argon  irradiation  adversely  affects  lesion  reflectance  mea¬ 


surements. 
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The  equipment  requirements  for  the  Retinal  Observation  and  Tracking 
Subsystem  have  been  carefully  detailed  throughout  this  document.  The  only 
requirement  that  may  conflict  with  the  Reflectance  Based  Feedback  Control 
System  is  the  need  for  a  visible  laser  beam  on  the  retina  to  recheck  terminal 
laser  position  after  position  update. 

Alternative  solutions 

There  are  several  different  methods  of  integrating  these  two  subsystems.  Al¬ 
ternatives  include  but  are  not  limited  to: 

•  Method  1:  Implement  two  distinct  subsystems  to  control  lesion  place¬ 
ment  and  depth.  These  subsystems  would  be  implemented  on  separate 
computer  hosts  using  separate  but  coaligned  imaging  systems.  This  tech¬ 
nique  was  proposed  by  Markow  [2]. 

•  Method  2:  Implement  a  single  system  to  accomplish  lesion  placement 
and  depth  control  using  a  semiconductor  diode  laser. 

•  Method  3:  Implement  a  single  system  to  accomplish  lesion  placement 
and  depth  control  using  an  argon  diode  laser. 

Method  1  provides  the  advantage  of  system  speed.  Implementing  sepa¬ 
rate  subsystems  allows  parallel  control  of  lesion  depth  and  placement.  However, 
this  method  has  inherent  disadvantages  including:  1)  duplication  of  expensive 
hardware  (high  speed  camera,  frame  grabber,  and  computer),  2)  the  need  for  a 
separate  master  controller  to  interface  the  two  systems,  and  3)  precision  optical 
alignment  of  the  two  systems. 
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Method  2  uses  a  single  system  to  control  both  lesion  placement  and 
depth.  This  method  solves  the  duplication  of  hardware  and  the  optical  align¬ 
ment  problems  of  Method  1.  The  diode  wavelength  (810  to  850  nm)  is  visible 
to  a  standard  CCD  video  camera  and  would  thus  interfere  with  reflectance 
measurements.  Use  of  a  single  system  to  control  lesion  placement  and  depth 
would  slow  overall  system  performance  since  these  two  tasks  would  be  per¬ 
formed  serially. 

Method  3  also  solves  the  duplication  of  hardware  and  the  optical  align¬ 
ment  problems  of  Method  1.  This  method  has  the  same  disadvantages  of 

Method  2. 

A  single  integrated  system  appears  to  be  an  attractive  answer.  How¬ 
ever,  several  key  questions  require  answers  prior  to  implementing  an  integrated 
system.  These  questions  include: 

•  Question  1:  Does  the  Retinal  Tracking  and  Observation  Subsystem  have 
the  required  resolution  to  monitor  lesion  growth? 

•  Question  2:  Are  camera  requirements  for  each  subsystem  compatible 
such  that  they  may  be  combined  in  a  single  camera? 

•  Question  3:  How  much  time  is  required  to  check  the  reflectance  re¬ 
sponse? 

•  Question  4:  Can  the  laser  irradiation  be  effectively  blocked  during  re¬ 
flectance  measurements? 

•  Question  5:  Can  lesions  be  formed  at  slower  rates  to  allow  testing  of  an 


integrated  prototype  system? 
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Both  Dr.  Jerath  and  I  believe  a  detailed  integrated  system  study  is 
the  next  logical  step  in  system  development.  The  goal  of  this  study  would  be 
answering  the  above  questions  and  implementing  a  single  Robotic  Laser  System 
to  precisely  control  both  lesion  depth  and  position. 

11.2.4  Neural  nets  to  learn  match  conditions 

Currently  loss  of  lock  is  determined  in  the  tracking  algorithm  by  comparing 
the  tracking  templates’  response  to  predetermined  upper  and  lower  thresholds. 
These  thresholds  are  determined  by  studying  template  response  from  both 
correct  and  incorrect  match  conditions.  As  described  in  Chapter  4  the  lower 
bound  was  set  at  0.15  of  the  expected  template  response  and  the  upper  bound 
was  set  at  1.3  of  the  expected  template  response.  The  determination  of  these 
thresholds  is  time  intensive. 

Artificial  neural  networks  may  be  used  to  determine  the  loss  of  lock 
thresholds.  Neural  networks  are  computer  models  of  the  behavior  of  real  neu¬ 
rons.  These  networks  are  made  up  of  three  types  of  units:  input  units,  output 
units,  and  the  hidden  units.  The  hidden  units  link  the  input  to  the  output 
units  via  weighted  links.  A  given  set  of  inputs  will  provide  an  output  should 
the  threshold  of  the  hidden  unit  be  exceeded.  Any  logic  function  may  be  mod¬ 
eled  with  neural  networks  by  proper  selection  of  weights  and  thresholds.  The 
weights  are  determined  by  the  neural  network  using  a  learning  set  of  inputs 
and  outputs  [113]. 

The  response  of  the  six  templates  used  in  the  tracking  algorithm  could 
be  provided  as  inputs  to  a  neural  net.  Proper  match  conditions  could  then  be 
learned  by  the  net.  The  output  of  the  net  would  be  a  lock/loss-ofJock  signal. 
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This  interesting  concept  requires  further  work. 

11.3  Summary  of  significant  findings 

This  research  effort  has  carefully  explored  the  use  of  digital  tracking  mech¬ 
anisms  to  correct  for  retinal  movement  during  laser  photocoagulation.  The 
following  is  a  summary  of  significant  findings  and  accomplishments: 

•  The  contrast  of  a  retinal  image  may  be  significantly  enhanced  by  com¬ 
bining  optical  filtering  techniques  with  frame  grabber  input  look-up  table 
modifications.  The  overall  effect  is  a  dramatic  increase  in  retinal  vessel 
contrast  against  the  retinal  background.  This  contrast  enhancement  may 
be  implemented  without  an  associated  time  penalty. 

•  Image  processing  techniques  were  conceptualized  and  developed  to  build 
lesion  data  bases  for  the  treatment  of  diabetic  retinopathy,  macular  de¬ 
generation,  and  retinal  breaks  and  tears. 

•  A  method  of  storing  multiple  retinal  fields  of  view  tracking  templates  and 
lesion  data  bases  was  conceptualized  and  developed. 

•  An  image  acquisition,  storage,  documentation,  and  enhancement  system 
was  developed  to  support  this  research  effort.  This  system  has  already 
been  used  to  support  other  research  efforts. 

•  A  tracking  algorithm  based  on  retinal  vessel  templates  has  been  imple¬ 
mented  which  uses  a  limited  exhaustive  search  to  determine  match  con¬ 
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•  A  development  system  was  designed  and  implemented.  This  development 
system  includes  a  retinal  imaging  system,  a  retinal  tracking  system,  a 
patient  interface,  and  a  laser  pointing  subsystem. 

•  Two  tracking  concepts  using  lesion  templates  were  developed,  modeled, 
and  tested. 

•  Closed  loop  feedback  control  of  laser  position  was  demonstrated  using 
the  tracking  algorithm  implemented  on  the  development  system.  This 
capability  was  demonstrated  with  calibrated  retinal  velocities  and  in  vivo 
using  a  rabbit  model. 

•  Panretinal  photocoagulation  and  the  repair  technique  for  retinal  tears 
and  breaks  were  demonstrated  in  vivo. 

•  Tracking  with  lesion  templates  was  demonstrated  in  vivo. 

•  A  trade-off  study  was  completed  to  determine  the  interrelationship  be¬ 
tween  the  factors  influencing  the  performance  of  the  tracking  algorithm. 

•  Results  of  the  trade-off  analysis  were  used  to  design  and  specify  the  re¬ 
quirements  for  the  real  time  system. 

11.3.1  Application  of  tracking  algorithm  to  other  laser  stabilization 
systems 

As  previously  mentioned,  some  of  the  imaging  software  developed  for  this 
project  has  already  been  used  to  support  other  research  efforts.  In  response  to 
this  need  a  general  purpose  imaging  software  package  based  on  the  Matrox  PIP- 
1024  and  the  Gateway  486-33  was  implemented  for  use  by  fellow  researchers. 
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A  laser  stabilized  against  subject  movement  is  a  common  problem  in 
retinal  photocoagulation,  corneal  thermokeratoplasty,  and  laser  tissue  and  ves¬ 
sel  welding.  Many  of  the  techniques  described  in  this  document  may  be  used 
in  these  other  related  areas  with  slight  modifications. 
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